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Foreword 


These  proceedings  were  printed  for  Headquarters,  U.S.  Army  Corps  of  Engi¬ 
neers  (HQUSACE)  and  the  work  performed  imder  Project  4A162784AT41,  “Mili¬ 
tary  Facilities  Engineering  Technology”;  Work  Unit  AP6,  “Design  Reviewer's 
Support  Environment.”  The  technical  monitors  were  Justin  Taylor,  CEMP-ES, 
and  Stan  Green,  CEMP-CE. 

The  workshop  was  hosted  and  the  proceedings  coordinated  by  the  Engineering 
Processes  Division  (PL-E)  of  the  Planning  and  Management  Laboratory  (PL), 
U.S.  Army  Construction  Engineering  Research  Laboratories  (USACERL).  The 
USACERL  principal  investigator  was  E.  William  East.  Dr.  Michael  P.  Case  is 
Chief,  CECER-PL-E,  and  L.  Michael  Golish  is  Operations  Chief,  CECER-PL. 
The  USACERL  technical  editor  was  Linda  L.  Wheatley,  Technical  Information 
Team. 


COL  James  T.  Scott  is  Commander  and  Dr.  Michael  J.  O’Connor  is  Director  of 
USACERL. 
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1  Introduction 


Background 

Many  efforts  have  been  made  to  develop  design  review  and  related  lessons- 
leamed  systems  throughout  the  U.S.  Army  Corps  of  Engineers.  Developers  and 
users  of  these  systems  expressed  an  interest  in  meeting  to  discuss  the  impact  of 
recent  technological  advances  in  network  computing,  nonproprietary  graphical 
presentation  of  contract  plans  and  specifications,  and  object-oriented  tech¬ 
nologies. 


Objective 

The  overall  objective  of  this  meeting  was  to  provide  a  forum  for  diMog  between 
developers  and  users  of  design  review  and  related  lessons-leamed  systems 
across  the  Corps.  This  dialog  should  improve  the  quality  of  individual  future 
products  and  reduce  potentially  duplicated  effort. 

The  specific  objectives  of  this  meeting  were  to  (1)  determine  the  state-of-practice 
in  design  review  and  related  lessons  learned  systems  and  (2)  to  identify  direc¬ 
tions  for  future  product  and  system  development. 


Approach 

The  approach  taken  in  this  workshop  was  to  allow  each  participant  to  describe 
their  role  in  the  design  review,  lessons  learned,  and  management  functions. 
Following  these  presentations,  breakout  sessions  were  held  to  cover  many  issues 
raised  during  the  presentations.  A  summary  meeting  following  the  breakout 
sessions  was  held  to  organize  the  conclusions  reached  on  topics  of  interest. 
During  this  summary  meeting,  the  group  developed  and  prioritized  a  set  of 
action  items. 


Scope 

This  report  docvunents  information  presented  at  the  January  1996  workshop 
held  at  the  U.S.  Army  Construction  Engineering  Research  Laboratories 
(USACERL)  to  determine  appropriate  future  directions  for  Biddability, 
Constructibility,  Operability,  environmental,  code  compliance  and  technical 
design  reviews;  and  the  application  of  lessons  learned  systems  for  design  review 
within  the  U.S.  Army  Corps  of  Engineers.  The  opinions  expressed  and  material 
provided  in  this  report  represent  the  best  knowledge  of  the  authors  at  the  time 
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of  that  meeting  and  are  not,  necessarily,  the  official  position  of  the  U.S.  Army 
Corps  of  Engineers. 


Mode  of  Technology  Transfer 

These  papers  have  been  published  on  the  World  Wide  Web  at  the  Design  Review 
Tools  Committee  Homepage,  http://www.cecer.army.mil/pl/ra/committee.  The 
conclusions  summarized  and  documented  in  the  final  paper  of  these  proceedings 
(p  67)  provide  guidelines  that  will  be  used  to  develop  future  design  review  and 
related  lessons  learned  products  and  systems. 
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2  Design  and  Construction  Feedback 
Systems 

by  Terry  Houghton^ 

Headquarters,  U.S.  Army  Corps  of  Engineers  (HQUSACE)  Directorate  of  Mili¬ 
tary  Programs  publishes  engineering  and  construction  policy,  guidance,  and 
criteria  documents.  They  also  recognize  that  there  must  be  a  continuing  evalua¬ 
tion  of  the  functional  responsiveness  and  technical  performance  of  the  Corps 
practices  for  design,  construction,  and  post-construction  for  constructibihty, 
engineering  and  technical  sufficiency,  life-cycle  cost  performance,  lessons 
learned,  technical  feedback,  and  compliance  with  current  design  and  construc¬ 
tion  criteria.  HQUSACE  has  established  various  requirements  and  methods  to 
obtain,  evaluate,  and  incorporate  recommended  changes  in  design  and  construc¬ 
tion  policy,  guidance,  and  criteria.  The  following  are  various  requirements  and 
methods  currently  employed  by  HQUSACE  to  obtain  constructive  feedback  for 
updating  their  policies,  criteria,  and  guidance  documents. 


Department  of  Army  Facility  Standard  Designs 

Engineer  Regulation  (ER)  1110-3-113,  “Department  of  the  Army  Facilities  Stan¬ 
dardization  Program”  addresses  the  following  review,  evaluation,  and  feedback 
requirements  for  the  Department  of  the  Army  (DA)  facility  standardization 
program: 

(a)  Approved  DA  standard  design  packages  are  monitored  and  evaluated  for 
responsiveness  to  user  requirements  and  for  technical  adequacy.  Revisions 
are  made  when  they  are  determined  appropriate  by  ongoing  review  and 
evaluation. 

(b)  A  subcommittee  for  each  standard  facility  type  is  responsible  for  evaluating 
the  responsiveness  of  the  DA  standard  design  package  to  the  user's  func¬ 
tional  and  operational  requirements.  The  subcommittee  monitors  facihties 
built  using  the  DA  standard  design  package,  evaluates  their  responsive¬ 
ness,  and  documents  the  findings. 

(c)  The  supporting  center  of  standardization  (COS)  for  each  facility  type  is 
responsible  for  evaluating  the  technical  performance  of  the  DA  standard 
design  package.  The  supporting  COS  monitors  facilities  built  based  on  the 
approved  DA  standard  design  package  (during  construction  and  post- 


’  Commander,  U.S.  Army  Corps  of  Engineers,  ATTN:  Mr.  Terry  Houghton  (CEMP-ET),  20  Massachusetts  Ave. 
NW,  Washington,  DC  20314-1000. 
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construction)  for  constructibility,  engineering  and  technical  sufficiency,  life- 
cycle  cost  performance,  lessons  learned,  technical  feedback,  and  compliance 
with  current  design  standards  and  construction  criteria.  The  supporting 
COS  documents  the  evaluation. 

(d)  The  subcommittee  and  the  supporting  COS  for  each  facility  type  coordinate 
their  reviews  and  evaluations  on  an  ongoing  basis.  As  a  minimum,  the 
groups  meet  once  a  year  and  provides  a  summary  of  their  actions  to  the 
Chairperson  of  the  DA  Facilities  Standardization  Committee.  The  sup¬ 
porting  COS  revises  the  DA  standard  design  package  when  required. 


Corps  of  Engineers  Guide  Specifications 

The  use  of  Corps  of  Engineers  guide  specifications  (CEGSs)  in  the  preparation  of 
project  specifications  is  mandatory  to  the  extent  the  guide  specifications  are 
applicable.  The  specifications  allow  contractors  to  provide  optional  materials 
and  methods  of  construction  that  are  of  a  type  and  quality  acceptable  for  mili¬ 
tary  construction,  as  a  means  of  increasing  competition  and  reducing  project 
costs.  Additional  options  may  be  considered  if  a  study  of  conditions  affecting  the 
project  shows  that  it  is  in  consonance  with  good  engineering  practice  in  that 
locality,  is  economically  justifiable,  and  is  in  the  best  interest  of  the  Govern¬ 
ment.  Where  cumulative  experience  indicates  that  a  change  in  standard  options 
may  be  advisable.  Division  commanders  are  asked  to  forward  their  recom¬ 
mendations  to  HQUSACE  (CEMP-EA)  using  ENG  Form  3078,  Recommended 
Changes  to  Engineering  Documents.  Also,  as  a  normal  use  of  CEGS,  ER  1110- 
345-720,  “Construction  Specifications,”  recommends  that  technical  or  editorial 
changes  that  are  necessary  or  desirable  for  general  application  or  to  adequately 
reflect  local  availability  of  materials  and  local  construction  practice  be  proposed 
via  ENG  Form  3078. 

HQUSACE  has  two  processes  by  which  CEGS  are  updated  to  current  industry 
standards  and  incorporates  recommended  changes  generated  by  field  personnel; 
the  Notice  Program  and  the  Criteria  Update  Program.  The  notice  program 
generally  addresses  minor  changes  that  can  be  accomplished  within  a  few  weeks 
time,  and  the  criteria  update  program  generally  addresses  major  changes 
requiring  extensive,  changes  and  takes  months  to  accomplish.  The  notice  pro¬ 
gram  updates  the  CEGS  on  a  two  month  to  one  year  schedule  and  the  criteria 
update  program  updates  the  CEGS  on  a  three  to  five  year  cycle.  Both  processes 
incorporate  the  ENG  Form  3078  recommended  changes,  HQUSACE  memoran¬ 
dums  and  HQUSACE-prepared  engineering  technical  letters,  feedback  obtained 
through  the  DA  standardization  program,  post  completion  inspections  and 
design/construction  evaluation  program,  and  industry  changes. 
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Technical  (Engineering)  Criteria 

Military  program  technical  engineering  criteria  are  published  in  the  form  of 
Army  regulations,  technical  manuals,  DA  facility  standard  designs,  engineer 
regulations,  engineer  pamphlets,  engineer  circulars,  engineer  technical  letters, 
architectm-al  and  engineering  instructions,  master  planning  instructions,  stan¬ 
dard  drawings,  definitive  drawings,  and  military  handbooks.  HQUSACE 
reviews  and  issues  updated  versions  of  the  above  documents  on  either  a  continu¬ 
ing  or  periodic  basis  (one  to  five  year  cycle).  At  the  time  these  publications  are 
updated,  they  incorporate  all  approved  ENG  Form  3078s  and  appropriate 
lessons  learned,  and  a  complete  industry  and  standards  update  is  conducted. 


Quality  Management  Reviews 

ER  1110-1-12,  “Quality  Management,”  requires  that  all  design  deficiencies, 
improvements,  and  field  changes  necessitated  by  missing  or  incomplete  design 
guidance/criteria  data  be  documented  and,  along  with  recommendations,  record¬ 
ed  and  submitted  to  HQUSACE  on  ENG  Form  3078.  HQUSACE  reviews  and 
incorporates  the  recommendations  into  the  criteria,  policy,  and  guidance  docu¬ 
ments  as  appropriate. 

HQUSACE  conducts  quality  management  reviews  of  each  Division  and  District 
on  a  three-year  cycle.  During  these  reviews,  many  lessons  learned  surface,  and 
HQUSACE  representatives  take  note  and  initiate  appropriate  actions. 


Design  and  Construction  Evaiuation  (DCE) 

ER  415-1-13,  “Design  and  Construction  Evaluation  (DCE),”  addresses  the  fol¬ 
lowing  review,  evaluation,  and  feedback  requirements  from  ongoing  military 
design  and  construction  projects.  This  ER  requires  HQUSACE  representatives 
to  conduct  annual  evaluations  of  ongoing  construction  projects  at  various  loca¬ 
tions  within  each  Division.  All  phases  of  construction  execution  are  examined 
for  compliance  with  contract  provisions  and  HQUSACE  guidance,  design- 
oriented  problems  are  investigated,  and  contract  documents  are  reviewed  for 
conformance  with  established  guidance. 

Upon  completion  of  each  field  evaluation  visit,  the  evaluation  findings  are 
reviewed  and  necessary  technical  findings  are  recorded  in  the  Construction 
Evaluation  Retrieval  System  (CERS)  and  maintained  at  CEMP-CE.  The  sys¬ 
tem's  file  is  utilized  for  feedback  and  periodic  distribution  of  common  problems 
to  the  field.  Comments  on  design  and  criteria  items  are  distributed  to  the  appro¬ 
priate  individuals  in  Engineering  Division  (CEMP-E)  for  necessary  revision  and 
addition  to  technical  manuals  and  guide  specifications.  The  inspection  team  is 
responsible  for  evaluating  criteria  deficiencies,  recommending  proposed  solu¬ 
tions,  and  providing  feedback  information  to  appropriate  HQUSACE  proponents 
for  consideration  in  initiating  policy  and  criteria  or  specification  improvements. 


12 


USACERL  CP-97/71 


Post  Completion  Inspection  and  Design  Criteria  Feedback  inspection 

ER  415-3-11,  “Post  Completion  Inspection  and  Design  Criteria  Feedback 
Inspection,”  addresses  the  following  review,  evaluation,  and  feedback  require¬ 
ments  from  completed  military  construction  projects.  This  ER  requires  post 
completion  inspections  conducted  to  identify  deficiencies  or  defects  in  design, 
construction,  materials,  equipment,  operability,  maintainability  or  functional 
adequacy  of  the  completed  facility.  These  inspections  are  conducted  approxi¬ 
mately  six  months  after  occupancy  of  a  facility. 

HQUSACE  design  guidance  must  be  kept  current,  in  part,  through  a  program  of 
periodic  inspections  of  facihties  that  have  been  in  use  for  two  or  more  years  or 
have  been  identified  as  having  design  criteria  or  functional  problems.  A  two- 
year-old  facility  which  has  been  user  tested  is  inspected  and  pertinent  com¬ 
ments,  recommendations,  description  of  facility  deficiencies,  evidence  of  poor 
serviceability  of  materials,  equipment,  or  operations  systems  are  recorded.  The 
inspection  team  is  responsible  for  evaluating  criteria  deficiencies,  recommending 
proposed  solutions  and  providing  feedback  information  to  appropriate 
HQUSACE  proponents  for  consideration  in  initiating  policy  and  criteria  or 
specification  improvements. 


Technical  Centers  of  Expertise 

Many  military-unique  engineering  and  construction  areas  are  not  readily 
available  from  the  private  sector  engineering  and  construction  consultants  and 
must  be  available  within  the  Corps  family.  To  maintain  many  of  these  unique 
engineering  and  construction  capabilities,  HQUSACE  has  assigned  centers  of 
expertise  responsibilities  to  designated  Major  Subordinate  Command  (MSC)  and 
District  commands.  These  centers  are  required  to  have  a  designated  xmique  or 
exceptional  technical  capability  in  a  specialized  subject  area,  many  of  which 
involve  emerging  or  rapidity  changing  technologies,  not  normally  foimd  but  very 
beneficial  to  other  military  program  USACE  commands.  All  other  MSC  and 
District  commands  are  instructed  to  coordinate  with  and  use  the  expertise  and 
services  of  the  centers  to  satisfactorily  accomplish  their  design  mission. 

ER  1110-3-109,  “Corps-Wide  Centers  of  Expertise  Assigned  to  Major  Subordi¬ 
nate  Commands  and  Districts”  covers  the  policy  and  responsibilities  for  military 
programs  mandatory  centers  of  expertise,  technical  centers  of  expertise,  and 
centers  of  standardization.  Center  responsibilities  include  (1)  assimilate  and 
analyze  lessons  learned,  (2)  provide  technical  feedback,  and  (3)  develop  recom¬ 
mendations  for  updating  and  revising  appropriate  design-construction  criteria, 
policy,  and  guidance  documents.  The  centers  recommend  proposed  solutions  to 
appropriate  HQUSACE  proponents  for  consideration  in  initiating  policy  and 
criteria  or  specification  improvements. 
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Cost  Engineering 


ER  1110-3-1300,  “Military  Programs  Cost  Engineering,”  requires  all  cost  engi¬ 
neering  elements  at  each  District  to  prepare  and  submit  awarded  construction 
cost  information  using  the  Historical  Analysis  Generator  (HAG)  cost  data 
reporting  system  to  HQUSACE  (CEMP-EC).  The  HAG  cost  data  are  then 
reviewed,  analyzed,  assembled,  consolidated  and  made  available  to  all  USAGE 
elements  and  other  services  on  an  electronic  bulletin  board  at  U.S.  Army  Engi¬ 
neering  and  Support  Center,  Huntsville  (CEHNC). 


Engineering  improvement  Recommendation  System  Builetin 

Iniprovement  Recommendation  System  (EIRS)  Bulletins  are  part  of 
the  process  for  implementation  of  recommendations  from  information  feedback 
sources,  and  are  used  in  military  construction  programs  to  facilitate  expedited 
dissemination  of  information  regarding  problems.  The  EIRS  Bulletins  are 
disseminated  monthly  to  all  MSC,  district  commands,  laboratories,  and  Field 
Operating  Agencies.  The  probable  solutions  included  in  EIRS  Bulletins  are  not 
thoroughly  explored  or  staffed.  As  such,  the  probable  solutions  will  not  repre¬ 
sent  a  final  HQUSACE  position,  and  their  use  is  not  mandatory.  Probable  solu¬ 
tions  are  considered  as  informational  in  nature  and  for  the  purpose  of  permitting 
prompt  consideration  by  the  field.  EIRS  Bulletin  recipients  (all  engineering 
offices)  are  encouraged  to  comment  on  the  probable  solution  presented  so  that 
other  viewpoints  can  be  considered  in  the  development  of  the  final  HQUSACE 
position. 


HQUSACE  Technical  Newsletters 

HQUSACE  publishes  newsletters  from  individual  techmcal  disciplines  address¬ 
ing  cost  engineering,  architectural,  mechamcal  engineering,  and  electrical  engi¬ 
neering  changes  in  criteria,  high  interest  topics,  and  associated  developments 
and  improvements.  Included  in  these  newsletters  are  lessons-leamed  topics 
related  to  each  technical  discipline. 


Staff  Assistance  Visits 

The  Corps  of  Engineers  Center  for  Public  Works  (CECPW)  conducts  staff  assis¬ 
tance  visits  to  Army  installations  and  provides  feedback  on  engineering  and 
construction  problems  they  have  encountered,  HQUSACE  initiates  corrective 
actions  as  required. 
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Corps-Wide  Technical  Conferences  and  National  Team  Meetings 

HQUSACE  sponsors  various  annual,  biannual,  and  triennial  Corps-wide  tech¬ 
nical  conferences  and  national  team  meetings.  During  these  conferences  and 
meetings  many  lessons  learned  surface,  and  HQUSACE  representatives  take 
note  and  initiate  appropriate  actions. 


MCA  Reprogramming  Actions 

HQUSACE  reviews  all  reprogramming  actions,  develops  lessons  learned  after 
performing  an  autopsy  of  failed  projects,  and  issues  these  lessons  learned  to  all 
Corps  offices. 


PROSPECT  and  CONTRAST  Training 

All  technical  disciplines  in  the  Headquarters  Engineering  Division  have  a 
number  of  PROSPECT  and  CONTRAST  training  courses  addressing  specific 
engineering  and  architectural  topics  associated  with  the  Corps  design  and  con¬ 
struction  program.  Each  course’s  instructional  material  is  updated  annually 
with  many  technical  engineering  lessons-leamed  items  incorporated  into  these 
updates. 
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3  Automated  Review  Management  System 
(ARMS) 


The  Automated  Review  Management  System 

by  Stephen  E.  Stoner^ 

Biddability,  Constructibility,  and  Operability  (BCO)  reviews  for  military  projects 
in  the  Sacramento  District  (SPK)  have  used  the  Automated  Review  Management 
System  (ARMS)  since  1988.  Engineering  Division  utilities  design  engineers  in 
its  Military  Design  Branch  use  ARMS  to  review  A/E  prepared  designs  and 
perform  peer  review  for  its  in-house  designs.  Engineering  Division  has  reduced 
its  level  of  review  detail.  In-house  peer  reviews  are  not  considered  technical 
reviews.  A/E  design  reviews  address  issues  such  as  life  and  safety.  Construc¬ 
tion-Operations  Division  uses  engineers  and  technicians  in  its  Construction 
Quality  Assurance  Section,  as  well  as  Resident/Area  Offices,  to  review  all  design 
packages  prepared  by  Engineering  Division. 

ARMS  is  used  by  Sacramento  District  reviewers  today  for  all  major  Civil  Works, 
Military,  and  Hazardous  Toxic  and  Radioactive  Waste  (HTRW)  projects.  Each 
reviewer  uses  PC  ARMS  for  Windows,  some  standalone,  while  others,  especially 
designers,  use  a  LAN-installed  version  to  respond  to  shared  comments.  Most 
using  agencies’  comments  are  uploaded  to  the  Oracle  7  based  ARMS  Central;  the 
remainder  of  user  comments  are  placed  in  the  ARMS  Central  program  by 
Technical  Managers. 

Currently  ARMS  implementation  barriers  are  primarily  found  in  our  using 
agency  or  customer  base.  The  agencies  with  limited,  infrequent  use  of  ARMS 
will  forward  written  comments  to  the  Technical  Managers  to  incorporate  into 
ARMS.  Information  on  the  local  and  notional  use  of  ARMS  can  be  found  in  the 
ARMS  usage  section  of  these  proceedings. 

The  Sacramento  District  military  design  process  from  final  submission  to  bid 
opening  was  investigated  by  a  Process  Action  Team  in  1995.  The  Team’s 
recommendations  included  continued  usage  of  ARMS.  The  PAT  did  recommend 
changes  to  the  design  process,  but  not  to  the  review  process.  The  Team  felt  that 
changes  in  the  design  process  would  be  the  most  effective  means  in  reducing  the 
cost  of  design  related  construction  change  orders. 

Capture  and  reuse  of  lessons  learned  in  the  Engineering  Division  uses  the 
Omaha  District  s  designed  and  developed  LAN  system  on  our  Sacramento 


Commander,  U.S.  Army  Engineer  District,  Sacramento,  650  Capitol  Mall,  ATTN:  Mr.  Stephen  E.  Stoner 
(CESPK-EDA)  Sacramento,  CA  95814-4794. 
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District  Banyan  LAN.  We  have  several  lessons  learned  from  each  discipline  now 
available  to  our  in-house  designers.  Each  design  section  is  responsible  for  the 
selection  and  update  of  their  discipline’s  listing  of  lessons  learned;  a  coordinator 
in  Otar  Criteria  Management  Unit  facilitates  inputting  the  lessons  learned  in  the 
database.  The  lessons-leamed  list  is  used  by  designers  during  the  early  stages 
of  design  to  assure  that  they  have  screened  their  projects  for  possible  repetitive 
errors.  Refer  to  the  Omaha  District’s  description  of  their  automated  lessons- 
leaomed  program  for  system  capabilities. 

An  initiative  to  distribute  electronic  bid  docoiments  for  review  and  other  pur¬ 
poses  was  initiated  by  SPK  in  1993,  with  a  follow  on  coordinated  effort  Avith  the 
South  Pacific  Division  (SPD).  To  date,  three  Sacramento  District  in-house  design 
projects  have  been  sent  electronically  to  our  SPD  offices  to  facilitate  their  QA 
review  of  our  in-house  MILCON  design  efforts.  This  district  is  now  in  the 
process  of  implementing  an  all  electronic  bid  set  deliverable  process. 

Engineering  Division  expects  that,  beginning  in  October  1996,  its  products  will 
be  advertised  in  an  all  electronic  format.  Review  and  comment  of  our  electronic 
products  will  primarily  use  .tiff  or  Adobe  Acrobat  formats.  The  determination  of 
which  format  to  use  will  depend  on  the  software  available  to  each  reviewer. 
Within  the  Sacramento  District,  50  copies  of  Watermark  Fax  software  are 
available  to  review  .tiff  files  using  its  markup  language.  Watermark  will  also  be 
used  to  receive  using  agency  comments  via  fax.  Multiple  copies  of  Acrobat  will 
also  be  available  for  both  viewing  and  marking  design  review  sets.  Which 
format  will  become  prevalent  will  be  determined  through  usage. 

Long  term.  Engineering  Division’s  electronic  products  could  be  integrated  into 
the  rapidly  developing  Internet.  As  our  contract  designer/construction  agents 
and  customers  connect  to  the  Internet  at  -t-lO  Mb  speeds,  all  of  us  could  be 
reviewing,  bidding,  and  constructing  Corps  projects,  programs,  and  services  by 
way  of  the  Internet.  Ooir  home  page  for  testing  these  concepts  is  http://www 
.usace.mil/cespk.html.  Comments  and  suggestions  regarding  our  thoughts  and 
efforts  are  welcome. 
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ARMS  Usage 


byJaeJ.Kim^ 


District  or  Division  Code 


CEHND 


CEMRK 


CEMRK 


CEMRO 


CENAB 


CENAN 


CENAO 


CENAP 


CENAP 


CENCB 


CENCD 


ENCS 


ENCS 


CENPA 


CENPP 


CENPS 


CENPS 


CEORL 


CEPOD 


CESAM 


CESAM 


CESAS 


CESAW 


CESPD 


CESPK 


CESPL 


CFSWA 


CESWF 


CESWL 


CESWL 


CESWT 


CESWT 


CETAD 


CEWFS 


OTHER 


Centrai  Site's  Computer  Name 


hnd41 


Total  Number  of  Projects  ■ 


’  Commander,  U.S.  Army  Engineer  District,  Sacramento,  650  Capitol  Mall,  ATTN:  Mr.  Jae  Kim  (CESPK-EDA), 
Sacramento,  CA  95814-4794. 
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An  Insight  Towards  Measuring  the  Design  Process 

by  Beth  A.  Brucker\  Michael  Golish®,  Astor  Ts.Ang® 

The  intent  of  this  study  is  to  find  ways  of  objectively  measuring  portions  of  the 
design  process  systematically.  The  Corps  of  Engineers,  as  indicated  in  Engi¬ 
neering  Regulation  1110-345-700,  divides  the  design  document  process  into  two 
parts:  concept  design  phase  (0  to  35%)  and  final  design  phase  (36  to  100%),  with 
design  document  reviews  required  at  35  and  95%.  The  first  phase  of  this  con¬ 
tinuing  study  is  the  examination  of  the  design  document  review  process.  Until 
recently,  capturing  and  managing  design  review  comments  at  these  different 
phases  was  ve^  hard  to  achieve.  In  the  Corps  of  Engineers,  reviewers  devel¬ 
oped  handwritten  design  comments,  often  at  scattered  locations,  with  limited 
interaction  with  other  reviewers  or  designers.  Often,  comments  were  not 
resolved,  causing  design  issues  to  extend  into  the  construction  process.  With  the 
fielding  of  the  U.S.  Army  Corps  of  Engineers  Automated  Review  Management 
System  (ARMS),  it  became  significantly  easier  to  manage  review  comments. 
Because  of  the  availability  of  review  comments  in  an  electronic  form,  a  content 
analysis  of  the  design  document  review  process  was  possible. 

Comments  from  ten  facilities  typical  to  the  commercial  construction  industry 
were  selected  from  the  Sacramento  District  of  the  U.S.  Army  Corps  of 
Engineers.  Both  35  and  95%  design  document  review  comments  from  each 
facility  were  coded  into  an  extensive  categorization  scheme  developed  for  this 
study.  The  categorization  scheme  classifies  each  comment  by  its:  location  in  the 
documents,  problem  type,  building  system  area,  and  related  disciplines.  The 
documents  reviewed  were  either  drawings,  specifications,  design  analysis, 
estimates,  or  other  required  Army  documentation.  Subcategories  of  the  problem 
type  address  issues  concerning  criteria,  design,  and  documentation. 

Once  the  comments  were  categorized,  percentages  of  review  comments  for  each 
category  were  determined.  The  most  frequent  problem  type  was  the  documen¬ 
tation  subcategory  for  both  35  and  95%  reviews.  Documentation  subcategories 
contain  issues  regarding  document  coordination,  omissions,  errors,  format,  pre¬ 
sentation,  and  terminology.  In  addition,  our  study  found  that,  of  the  building 
systems  classified,  mechanical,  structural,  sitework,  and  envelope  are  respon¬ 
sible  for  the  majority  of  total  incidents.  This  finding  is  similar  to  a  previous 
study  of  professional  liability  cases  on  construction  failures  conducted  by  the 
University  of  Maryland's  Architecture  and  Engineering  Performance  Informa¬ 
tion  Clenter  (AEPIC). 

The  next  phase  of  this  study  is  to  collect  the  change  order  documentation  of  the 
ten  facilities  previously  analyzed.  It  is  hypothesized  that  a  comparison  of  the 


‘Commander,  U.S.  Army  Construction  Engineering  Research  Laboratory,  ATTN:  Ms.  Beth  Brucker  (CECER-PL- 
E),  P.O.  Box  9005,  Champaign,  IL  61826-9005. 

'Commander,  U.S.  Army  Construction  Engineering  Research  Laboratory,  ATTN:  Mr.  Michael  Golish,  (CECER- 
PL),  P.O.  Box  9005,  Champaign,  IL  61826-9005. 

‘  Commander,  U.S.  Army  Constnjction  Engineering  Research  Laboratory,  ATTN:  Ms.  Astor  Ts.Ang  (CECER-PL- 
E),  P.O.  Box  9005,  Champaign,  IL  61826-9005. 
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analysis  of  the  design  reviews  to  an  analysis  of  change  orders  of  each  project 
may  show  the  benefits  and/or  deficiencies  in  the  design  review  process.  As  a 
final  strategy,  it  is  hoped  that  a  post-occupancy  evaluation  could  give  holistic 
insight  towards  measuring  the  design  process. 


Data  Analysis  Sheet  Notation 

Comment  Location 

1.  Drawings 

2.  Specifications 

3.  Analysis  of  Design 

4.  Estimate 

5.  Other  documentation  (e.g.,  DD1391,  DD1354,  RFP,  VEP,  etc.) 

Disciplines 

Standard  List  of  ARMS  disciplines 

Comment  Type 

1.  Mandatory  Change/Requirement 

2.  Recommendation 

3.  Verification 

4.  Justify  /  Explain 

5.  Question 

6.  Reminder  /  Not  needed  at  this  phase  /  Next  submission 

7.  Concurrence 

8.  Info 

9.  Duplicate 

10.  Reviewer  Error 

Problem  Type  (4  character  field — ^use  0  if  none) 

1  Criteria 

11  Using  wrong  criteria  /  use  other  criteria 

12  Not  in  Compliance  with  requirement/criteria 

13  Not  using  current  design  criteria/source  info  (CEGS) 

2  Design  Issues 

21  Scope 

211  Delineate  scope  of  work  (e.g.,  construct  boundaries,  extent,  demolition) 

212  Not' in  contract  scope  /  change  in  scope 

213  Missing  feature  /  requirement 

214  Unnecessary  feature  /  requirement 

215  Excessive  use  of  feature  /  reqt.  (reduce  #) 

216  Inadequate  use  of  feature  /  reqt.  (increase  #) 

217  Unenforceable  requirement  (not  feasible) 

218  Assignment  of  Work  (subcontr  single/  prime) 

22  Design  analysis 

22 1  Questionable  /  incorrect  assumptions 
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222  Increase  /  decrease  performance  of  featvire 

223  Not  customary  /  local  practice 

224  High  maintenance  /  operations  solution 

225  Inadequate  analysis 

23  Design  configuration 

231  Change  Design  Solution  (too  complex,  doesn't  meet  req.) 

232  Size  feature  (size  parking  stall,  transformer,  AHU) 

233  Routing  or  location  of  feature  (control  joints) 

234  Clearances  /  soft  and  hard  interferences 

235  Match  existing  featm*es  (e.g.,  match  keying  schedule) 

236  Relocation  of  existing  feature 

237  Use  Standard  detail 

238  Use  Standard  Master  Spec  (CEGS) 

24  Product  /  System  selection 

241  Feature  not  an  option  in  CEGS 

242  Inappropriate  or  illegal  for  local  conditions  /  project 

243  Inappropriate  for  Application 

244  Not  cost  effective  selection 

245  Options  overly  restrictive 

246  Select  material/product/system  with  different  performance/characteristics 

25  Coordination 

251  Systems/product  interface/integration 

252  Phasing  of  construction 

253  Disposed  of  waste  materials 

254  Salvage/Reuse 

255  Conflicting  info 

3  Documentation 

31  Coordination 

311  Conflicting  information/?  combine 

312  Document  Coordination/?  combine 

3121  Drawing/specs 

3122  Drawing/drawing 

3123  Specs/specs 

3124  Draw  /  Support  documents 

3125  Specs  /  Support  documents 

3126  Estimate  /Other  documents 

32  Omissions  (-.-0)/  errors  ( — 1) 

321  Missing  (or  need  additional)  information/  errors 

322  Missing  detail  /  drawing  /  schedule/  errors 

323  Missing  specification  text  /  errors 

324  Missing  S3anbol,  reference,  annotation,  labels  /  errors 

325  Missing  calculations  or  support  documentation 

326  Dimension  errors  or  missing  dimensions 

327  Not  applicable  to  project,  imnecessary  info  (spec  pzu*a) 

33  Format  /  Presentation  of  Information 

331  Change  format/orientation  (note  to  schedule) 

332  Compliance  with  specification,  drawing,  cost  formats 
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333  Notes/notation/symbology/labeling/  annotations 

334  Location  of  information  (spec  sect,  sheet  #) 

335  Readability  (line  weight,  lettering  too  small) 

34  Terminology 

341  Use  of  obsolete  /  inappropriate  terms 

342  Confusing  /  awkward  phrasing  /  grammar 

343  Spelling  /  typo  /  missing  text 

344  Use  of  inappropriate  phrase  (owner/  government) 

345  Use  of  proprietary  specifications,  references,  notes 

35  Estimates 

351  Inadequate  detail  (e.g.,  split  into  floors,  wings) 

352  Labor  rates  /  productivity  factors 

353  Quantity  errors  /  time  estimate  errors 

354  Unit  costs  errors 

355  Overhead  /  profit  errors 

356  Contingency  errors 

357  Midpoint  of  construction  error 

358  Price  quote  needed 

359  Total  Cost  error 

CEG  Work  Breakdown  Structure 

Not  Applicable,  unknown 

Substructure 

Structure 

Roofing 

Exterior  Closure 
Interior  Walls 
Interior  Finishes 
Specialties 
Plumbing 
HVAC 

Special  Mechanical  Systems 
Electrical 

Special  Electrical  Systems 
Equipment  and  Conveying 
Site  Preparation 
Site  Improvements 
Site  Utihties 

General  Rqmt,  Contract  Rqmt,  Bidding  Info 
Multiple  Systems 

Source  Requirement 

Reviewer  determination/Unknown 
Code  or  Regulation  (NFPA) 

Design  Criteria,  Corps  Policy  (TM,  ER,  ETL,  AE  instruct) 
District  Policy  (A/E  Guide,  AFM,  AFR) 

User  Request  /  requirement 
Industry  practice 


22 


USACERL  CP-97/71 


4  Developments  at  Corps  Districts 


Huntsville  Division's  Survey  of  Corps  Systems 

by  Linda  Himmelright’ 

CEHND  (U.S.  Army  Engineer  Division,  Hiintsville): 

Chem  Demil  -  Request  for  Information  (RFI),  Engineering  Change  Proposal 
(ECP)  (HND-draft;  contractor-master)  (Dale  Campbell-205-895-1765) 

AE  Contracts  -  LL;  not  database;  new  procedures  (Terry  Bm*ton-205-895- 
1381) 

ED-ES-G  -  Guide  Spec  Bulletin  Board  (Jim  Quiim-205-895-1821) 

CELMN  (U.S.  Army  Engineer  District,  New  Orleans):  Construction  contract 
review  (Robert  Guillot-504-862-2938) 

CEMRO  (U.S.  Army  Engineer  District,  Omaha):  HTRW  LL  (Claudia  Wiethop- 
402-697-2561)  Engineering  Division  LL  (Nadir  Khan-402-221-4915) 

CEORD  (U.S.  Army  Engineer  Division,  Ohio  River):  no  work  so  far  but  will  be 
using  Mobile  (John  Hart-513-684-3803) 

CEORH  (U.S.  Army  Engineer  District,  Huntington):  HTRW  uses  MRO  HTRW 
(Brent  Smith-304-529-5640)  Construction  reviewed  Mobile  (concern:  lacking 
user-fiiendliness) 

CEORL  (U.S.  Army  Engineer  District,  Louisville):  LL;  Engineering;  obtained 
from  Omaha  District  (Jack  Skinner,  Value  Engineering  Ofcr-502-582-6058) 

CESAM  (U.S.  Army  Engineer  District,  Mobile):  LL;  joint  Engineering/Construc¬ 
tion  (Glenn  Howard-334-690-3447) 

CETAC  (Corps  of  Engineers  Transatlantic  Programs  Center):  Plans  and 
Operations  (Carroll  McDonald-IM) 


^  Commander,  U.S.  Army  Engineer  Division,  Huntsville,  P.0,  Box  1600,  ATTN:  Ms.  Linda  HImmelright  (CEHND- 
TD).  Huntsville.  AL  35807-4301 . 
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Omaha  District’s  Use  of  ARMS 

by  Margie  J.  Cnimley® 


Background 

I  am  Margie  Crumley,  and  I  am  the  ARMS  Coordinator  for  the  Omaha  District. 
ARMS  is  an  acronym  for  Automated  Review  Management  System  and  is  the 
application  that  I  work  with. 

My  position  at  the  Omaha  District  is  as  an  Engineering  Tech.  The  position  was 
created  in  the  late  80s  to  work  with  automation  of  reviews  and  to  assist  techni¬ 
cal  managers,  then  known  as  project  managers,  with  getting  ready  for  review 
conferences. 

The  Omaha  District  at  that  time  was  one  of  the  districts  CERL  was  working 
with  on  the  development  of  ARMS.  My  understanding  is  that  then  the  com¬ 
ments  being  entered  needed  to  be  done  on  a  main  frame,  and  Omaha  District 
reviewers  did  not  have  the  connectivity  that  would  have  been  required.  At  that 
point,  probably  1988,  it  was  decided  that  Omaha  District  would  bring  on-board  a 
programmer  to  write  a  program  that  could  be  used  on  an  individual  PC  to  do 
comments,  and  the  program  woiild  be  compatible  with  the  ARMS  program 
CERL  was  developing.  The  intent  was  to  then  upload  the  comments  that  were 
generated  dimng  the  period  to  the  main  frame  used  for  ARMS,  after  CERL 
developed  this  capability. 

The  Omaha  District  program  was  used  on  individual  PCs  initially  and  then 
moved  to  LANs.  It  was  also  furnished  on  disk  to  Architect-Engineers,  and  used 
by  oim  in-house  designers  to  make  responses.  The  programming  for  Omaha 
District  was  done  in  Clipper,  and  the  program  that  Omaha  District  ended  up 
^th  was  called  COMNET .  Interestingly,  the  same  programmer  developed 
“Omaha's  Lessons  Learned  System”  that  you  will  hear  about  later.  The  Lessons- 
Leamed  program  uses  numerous  modules  from  the  COMNET  program. 

In  1990,  when  USACE  decided  to  implement  ARMS  Corps- wide  for  reviews  on 
Military  projects  and  set  up  the  TCX  at  Sacramento  to  do  training  at  the 
Divisions  and  Districts,  Missoviri  River  Division  (MRD)  contracted  with  our 
programmer  to  work  on  an  interface  of  our  program  with  ARMS.  Although  work 
on  this  interface  ended  when  MRD  decided  to  move  over  to  ARMS,  I  was  part  of 
the  programming  on  the  interface,  and  we  did  use  it  to  send  comments  to  ARMS 
from  our  program  imtil  ARMSWord,  as  the  reviewer  package  for  ARMS  was 
known  at  that  time,  was  made  network  compatible  by  the  programmers  at 
Sacramento.  This  did  not  happen  until  after  the  first  ARMS  User  Conference  in 
1991. 


*  Commander,  U.S.  Army  Engineer  District,  Omaha,  ATTN:  Margie  Crumley  (CEMRO-ED-MG),  215  North  17" 
Street,  Omaha,  NE  68102-4978. 
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Organization 

Personnel 

The  Technical  Managers  (TMs)  of  the  Design  Branch  all  work  with  ARMS.  The 
Environmental  Branch  TM's  who  work  with  Military  projects  also  work  with 
ARMS  to  some  degree.  On  other  Environmental  Branch  projects,  the  project  is 
initialized  on  Engineering  Division  Network,  where  PCARMS  is  loaded.  If  oiu* 
Omaha  District  reviewers  are  making  comments  on  the  design  documents — ^the 
request  to  have  the  project  initialized  often  comes  from  our  own  reviewers,  who 
are  used  to  doing  comments  using  PCARMS.  Some  of  the  project  managers  from 
the  PPM  Division  also  use  ARMS,  but  the  request  for  ARMS  use  is  generally 
made  by  someone  outside  the  District  that  has  used  ARMS.  We  work  with  a  few 
other  TMs  outside  of  Design  and  Environmental  Branches  of  the  Engineering 
Division — Geotech  Branch,  Operations  Division,  Planning  Branch. 

The  reviewers  in  the  Omaha  District,  with  the  exception  of  two  Environmental 
Branch  sections  who  have  limited  participation,  are  required  to  use  ARMS.  The 
projects  are  set  up  on  the  Engineering  Division  Network,  and  the  reviewers  who 
are  not  comfortable  with  doing  comments  there,  because  of  an  occasional  loss  of 
comments  and  having  other  reviewers  in  the  file  while  they  are  there,  have 
PCARMS  loaded  on  the  hard  drive  of  their  computer  and  do  their  comments 
there.  When  they  are  finished,  their  comments  are  uploaded  to  the  network. 
Although  Omaha  District  has  numerous  LANs  and  the  PCARMS  software 
resides  on  the  Engineering  Division  network,  our  network  individuals,  and  in 
particular  Drew  Anderson,  have  made  a  concerted  effort  to  have  anyone  who  has 
reason  to  access  Engineering  Division  Network  to  be  able  to  do  it.  The  LAN  for 
Environmental  Branch,  which  is  not  in  our  building,  is  maintained  by  our  IM 
personnel  and  also  has  full  access  to  our  network. 

Projects 

The  August  1995  edition  of  the  ARMS  Newsletter  shows  Missouri  River  Division 
with  1,320  projects  implemented  on  the  MRK41  computer.  This  is  considerably 
more  projects  than  any  other  Division,  outside  of  Sacramento,  and  not  that 
many  less  than  Sacramento.  Approximately  75%  of  those  projects  are  Omaha 
District’s. 

Automated  Revtew  Management  System  Deveiopment 

The  rationale  behind  ARMS  was  to  have  all  the  players  in  the  review  arena  be 
able  to  access  a  central  location  to  upload  comments  and  responses  for  a  given 
project.  The  capability  for  tracking  projects  was  also  available.  When  USAGE 
mandated  that  ARMS  be  used  for  Military  projects  in  1990,  it  was  considered 
conceivably  that  Civil  and  HTRW  projects  would  eventually  be  reviewed  using 
ARMS  also. 

Omaha  District’s  move  to  the  MRK41  computer  came  in  April  of  last  year.  The 
MRK41  computer  is  much  faster  in  doing  emything  that  is  required  on  the  Dell 
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computer  that  was  used  as  the  central  computer  and  located  at  Omaha  District. 
Although  Oracle  7  is  on  the  MRK41  computer,  Missouri  River  Division  has  not 
moved  over  to  the  Oracle  version  of  ARMS  yet.  The  biggest  reason  why  is  the 
need  for  the  ARMS  coordinator  to  be  able  to  work  with  the  Oracle  file  when 
there  are  problems,  as  I  am  able  to  do  now  through  the  UNIX  operating  system. 
Sacramento  is  working  through  a  program  that  they  have  written  to  give  Omaha 
District  that  capability,  and  hopefully  we  will  be  moved  over  soon.  The  ability  to 
work  with  the  file  is  particularly  crucial  since  the  funding  for  the  TCX  has  been 
cut  back. 

Omaha  District  Usage 

In  my  position,  I  work  with  all  levels  involved  in  the  review  process  for  ARMS.  I 
have  developed  instructions  for  working  with  and  without  modems  for  reviewers 
and  Architect-Engineer  firms  working  with  us,  and  they  are  furnished  these 
instructions,  along  with  my  telephone  number  if  they  need  help,  when  they  are 
working  with  us. 

Omaha  District  has  made  a  concerted  effort  to  have  installations  as  well  as 
other  areas  working  with  ARMS.  Having  worked  with  ARMS  from  the  time  that 
Omaha  District  moved  over  to  ARMS,  I  feel  that  the  concept  of  ARMS  is  a  good 
one,  and  should  help  where  reviews  are  concerned.  I  have  in  my  files  copies  of 
review  comments  that  are  hand  written,  in  some  cases  extremely  difficult  to 
decipher,  and  can  remember  the  technical  managers  pulling  the  handwritten 
comments  together  by  disciphne  for  a  review  conference. 

Construction  Division  at  Omaha  District  has  our  field  offices  imder  them  and 
has  been  particularly  helpful  on  pushing  for  comments  being  submitted  on 
ARMS.  The  flip  side  of  that  coin  is  assmrance  that  annotations  to  their  field 
offices'  comments  will  show  up  on  ARMS  in  a  timely  manner. 

The  way  ARMS  is  set  up  to  work,  the  TM  goes  to  the  central  computer  and 
initializes  a  project.  At  Omaha  District  I  generally  do  it,  after  receiving  a  copy  of 
the  memo  asking  for  comments.  Next  he  sets  up  the  phase  being  reviewed  and 
then  chooses  the  routing  schediile.  The  routing  needs  to  be  to  a  login  that  is 
recognized  by  the  ARMS  central  program.  And  just  because  you  have  a  login  for 
one  central  computer,  you  will  not  be  able  to  work  on  another  Division's  central 
computer  until  the  login  is  established  there.  This  will  not  be  the  case  when 
everyone  is  working  with  the  Oracle  version — you  will  be  able  to  work  anywhere 
you  need  to.  At  this  time,  Missouri  River  Division  is  still  working  with  the  old 
version;  hopefully  we  will  get  moved  over  to  Oracle  7  soon. 

The  TM  routes  to  a  login  that  is  set  up  for  either  a  review  manager  or  a 
reviewer.  If  he  routes  to  a  review  manager,  the  review  manager  will  need  to 
access  the  central  computer  and  route  the  project  to  the  reviewers. 

At  this  point,  the  reviewer  does  not  care  about  the  routing.  The  reviewer  has 
received  a  memo  from  the  TM  which  probably  forwarded  the  design  docxunents 
and  asked  for  comments  back  by  a  certain  date.  In  the  case  of  the  Omaha 
District,  we  ask  that  there  be  a  standard  paragraph  in  the  memo,  advising  that 
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the  project  has  been  initialized  on  ARMS,  and  asking  that  the  comments  be  done 
using  ARMS  if  possible. 

The  reviewer  now  sets  up  the  project  on  his  individual  computer  or,  if  he  is 
working  on  a  network,  he  chooses  the  applicable  project  (at  Omaha  District  I  set 
the  projects  up  on  the  network)  and  does  the  review  comments.  After  the 
comments  eu^e  finished,  if  the  reviewer  has  his  own  login  and  can  get  to  the 
central  computer,  either  through  Procomm  (the  software  that  is  furnished  with 
the  PCARMS  software  and  dials  in)  or  through  the  “telnet”  capability,  if 
available,  he  accesses  the  central  computer  and  uploads  his  comments  back  to 
the  TM. 

At  Omaha  District  we  do  not  have  that  capability  for  our  reviewers  at  this  time, 
and  the  reviewers  share  a  login — depending  on  what  area  they  work  in  (i.e.. 
Design  Branch).  Instead,  after  the  comments  on  a  project  are  finished,  I  will  go 
ahead  and  export  them  from  the  network  as  an  ASCII  file  and  upload  them  to 
the  central  computer. 

After  the  comments  are  uploaded  from  Engineering  Division  Network  back  to 
the  TM,  they  are  combined  with  comments  that  have  been  uploaded  from  oiir 
field  offices  or  other  installations  that  work  with  us,  and  the  TM  gets  a  printout 
of  the  comments  for  the  review  conference.  At  a  recent  review  on  a  facility  at 
Buckley  ANG,  we  received  better  than  700  comments  from  the  reviewers  at 
Buckley.  The  people  at  Buckley  had  problems  years  ago  with  uploading  their 
comments  to  us  electronically  and  still  opt  to  “overnight”  a  disk  with  their 
comments. 

Of  conceivably  a  more  critical  nature,  our  field  offices  are  able  to  get  comments 
to  us  on  Bid  Documents  in  a  matter  of  minutes,  and  this  has  us  looking  at  fewer 
amendments  or  modifications  at  the  bid  stage. 

The  final  “leg”  on  the  review  process  at  Omaha  District  is  to  send  the  annotated 
comment  file  to  the  designer,  either  oiir  in-house  people  or  the  Architect/Engi¬ 
neer.  The  designer  then  enters  responses,  indicating  where  changes  were  made 
in  the  design  documents,  and  returns  them — either  electronically  or  on  a  disk. 
The  comments  with  the  responses  then  become  part  of  the  Design  Analysis  for 
the  next  phase. 

Barriers  to  Implementation  and  Solutions 

The  manner  in  which  Omaha  District  is  set  up  to  work  on  the  network  with 
ARMS  has  caused  some  problems  with  reviewers  losing  comments.  We  do  have 
a  backup  file  that  comments  feed  into  when  a  reviewer  leaves  PCARMS,  and  the 
file  can  be  used  to  retrieve  comments  if  they  are  lost — sometimes.  We  have 
PCARMS  set  up  the  way  Sacramento  was  instructing  us  to  when  we  first  started 
working  on  the  network,  and  their  instructions  are  now  different.  Instead  of 
having  all  the  reviewers  in  one  file  out  on  the  network  as  we  do,  Sacramento 
now  has  them  working  off  their  individual  PCs-  and  only  using  the  network  to 
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upload  their  comments  after  they  have  finished  them.  I  believe  this  is  probably 
better,  and  when  I  am  asked  about  setting  PCARMS  up  on  a  network,  I  send 
instructions  for  working  both  ways  but  suggest  using  the  method  Sacramento  is 
using  now. 

We  have  not  been  able  to  move  away  from  “hard  copies”  of  review  comments  as 
we  had  hoped.  I  went  out  to  Fort  Carson  for  a  review  conference  in  1994.  We 
took  out  limited  copies  of  the  review  comments  and  a  notebook  computer,  with 
the  idea  of  having  the  comment  projected  from  the  computer  screen  to  a  wall.  I 
was  going  to  do  the  annotations.  The  notebook  computer  that  we  used  did  not 
seem  to  be  able  to  project  the  comment  clearly  enough  for  about  half  of  the 
reviewers  to  see  it,  so  we  defeated  our  purpose  by  having  to  read  the  comments. 
We  moved  to  a  better  sized  room  for  the  conference  the  second  day,  and  then  the 
notebook  computer  gave  us  trouble,  and  having  no  backup,  the  reviewers  ended 
up  having  to  share  the  few  hard  copies  of  the  comments  that  were  available. 

Omaha  District's  big  push  at  the  first  user's  conference  in  1991  was  to  task  the 
TCX  (Technical  Center  of  Expertise)  with  making  the  ARMSWord  software,  as  it 
was  called  at  that  time,  network  compatible.  This  was  done. 

The  Sacramento  TXC  personnel  were  at  Omaha  District  to  train  personnel  to 
work  with  ARMS  in  1991,  and  those  individuals  who  were  trained  were  to 
become  trainers.  In  actuality,  I  do  the  classes  for  the  Technical  Managers  and 
the  Reviewers,  rather  than  the  trainers;  the  reviewers  who  were  trained  by 
Sacramento  could  probably  have  answered  questions  initially.  Unfortunately, 
by  the  time  we  had  done  heta  testing  on  the  network  compatible  ARMSWord  and 
were  actually  using  it,  too  much  time  had  elapsed  for  the  training  to  be 
meaningful. 

The  TMs  also  had  problems,  initially,  with  connectivity  to  the  central  computer. 
All  that  was  available  was  Procomm,  and  access  to  it  was  limited  to  sharing  a 
“hard”  line  with  usually  three  other  individuals,  including  the  time  keeper.  We 
now  have  telnet  capability  and  easy  access  since  the  TMs  have  been  set  up  with 
a  batch  file  that  automatically  feeds  in  their  IP  number,  and  moves  them  to  the 
MRK41  computer  at  Kansas  City,  which  we  are  currently  using  for  ARMS. 

At  one  time,  annotations  to  the  comments,  usually  made  at  a  review  conference, 
had  to  be  done,  out  on  the  central  computer.  This  did  not  turn  out  to  be  a  good 
situation;  the  TMs  lost  annotations  if  they  did  not  leave  the  file  often  and  “save.” 
Consequently  the  TCX  personnel  were  asked  at  our  second  ARMS  Users’ 
Conference  to  give  us  the  ability  to  download  a  file  for  annotations,  and  that  was 
accomplished. 

We  have  not  had  that  much  “push”  fi”om  higher  Headquarters  recently  on  using 
ARMS  and,  when  that  happens,  it  seems  that  any  problem  will  be  used  as  an 
excuse  not  to  use  ARMS.  The  number  of  Military  projects  are  down  considerably 
for  this  fiscal  year,  although  they  are  supposed  to  be  up  again  in  1997. 
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Future  Direction 

At  Omaha  District,  the  PCARMS  software  that  is  loaded  on  the  Engineering 
Division  network  is  DOS  based.  We  have  done  beta  testing  on  the  PCARMS  for 
Windows  software  that  Sacramento  District  has  developed,  and  now  that  the 
software  is  ready  for  use  (December  1995),  it  has  been  loaded  on  our  network 
and  the  reviewers  will  be  offered  a  choice  of  working  with  either  the  DOS  or 
Windows  version. 

Although  we  have  the  PCARMS  software  on  the  Engineering  Division  Network 
at  Omaha  District,  Drew  Anderson  has  worked  to  make  it  possible  for 
individuals  from  other  networks  to  access  Engineering  Division  Network  and 
make  comments.  In  addition  to  Design  Branch,  we  have  input  from  reviewers  in 
Construction  Division,  Planning  Division,  Environmental  Branch,  Geotech 
Branch,  etc. 

I  am  excited  about  the  direction  PCAEMS  has  gone  where  Windows  is  con¬ 
cerned.  It  seems  that  our  technical  managers  may  be  more  receptive  to  using 
ARMS  through  Windows  than  the  DOS-based  version.  One  reason  for  this  is 
that  it  is  easier  when  they  are  already  working  in  Windows  not  to  have  to  leave 
the  application. 
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Omaha  District's  Pians  for  Using  Eiectronic  Bid  Documents 

by  Drew  Anderson® 


Con<xpt 

(a)  Programs  and  Project  Management  (PPM)  would  create  a  new  project  in  the 
Electronic  Contract  Bid  Management  System  (ECBMS). 

(b)  Include  information  such  as  the  Standard  Industrial  Classification  (SIC), 
description,  and  dates. 

90%  Review 

(a)  Engineering  would  prepare  bid  documents  using  Adobe  Acrobat  software  for 
text  documents  and  CALS  format  for  drawings  and  load  them  into  ECBMS. 
(CALS  is  DOD  standard  for  raster  drawings.) 

(b)  ECBMS  will  require  a  file  description  of  each  file  placed  in  the  directories. 
This  information  will  be  used  to  create  a  Table  of  Contents. 

(c)  Reviewers  could  view  plans  and  specs  either  on-line  or  via  CDs. 

(d)  Contracting  would  review  the  front  end  specifications. 

Final 

Engineering  and  Contracting  would  update  their  bid  documents  in  ECBMS. 

Advance  Notice  to  Bidders 

(a)  Specifications  sends  Contracting  the  CBD  announcements. 

(b)  Contracting  will  mark  the  project  for  “Advance  Notice  to  Bidders”. 

(c)  The  Web  Site  will  then  show  the  project  but  not  allow  contractors  to  down¬ 
load  files. 

(d)  The  Central  Web  Site  would  be  updated. 

Authority  To  Advertise 

(a)  PPM  informs  Contracting  that  authority  to  advertise  has  been  given  by  the 
user. 

(b)  Contracting  updates  ECBMS. 

(c)  Electronic  Bid  Documents  would  be  made  Read/Only. 

(d)  Bid  Packages  would  be  sent  to  registered  bidders. 

(e)  The  Web  site  would  now  show  the  project  as  active. 

(D  The  Central  Web  site  at  WES  would  be  updated. 

(g)  FileS'  would  be  copied  to  the  FTP  site  for  download. 


’Commander,  U.S.  Army  Engineer  District,  Omaha,  ATTN:  Drew  Anderson  (CEMRO-ED-DI),  215  North  17" 
Street,  Omaha,  NE  68102-4978. 
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Amendment 

(a)  Specifications  would  issue  an  amendment  to  Contracting. 

(b)  Contracting  would  e-mail  the  amendment  to  all  registered  bidders.  (Need  to 
verify  that  the  bidder  received  the  amendment) 

(c)  Amendments  would  be  e-mailed  to  all  registered  bidders.  (Anyone  ordering  a 
CD  after  amendments  have  been  issued  would  receive  e-mail  about  those 
amendments.) 

Bid  Opening 

(a)  Contracting  indicates  that  bids  are  being  accepted. 

(b)  FTP  Server  IDs  are  enabled. 

BidCiosit^ 

(a)  FTP  Server  IDs  are  disabled. 

(b)  Contracting  would  gather  bids  firom  the  FTP  server  and  prepare  results. 

Award 

(a)  Contracting  would  input  the  bid  results  in  ECBMS. 

(b)  The  Web  Site  would  display  these  results. 

(c)  The  Central  Web  site  at  WES  would  be  updated. 

(d)  Bidders  would  be  e-mailed  bid  results. 

Construction 

PMs,  TMs,  CADD,  Specs,  Construction,  and  Contracting  will  place  documents 
that  pertain  to  the  project  in  ECBMS.  (Need  an  SOP  on  what  documents 
should  be  stored  and  in  what  format.) 

Finai  payment 

(a)  Contracting  closes  out  the  project. 

(b)  CDs  will  be  created  and  sent  to  Contracting,  Construction,  Engineering,  and 
the  customer. 
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Omaha  District's  Lessons-Learned  Process 

by  Drew  Anderson 

Organizational  Context 

Process  primarily  involves  Engineering  and  Construction  Divisions  (approxi¬ 
mately  600  people).  Process  is  available  and  is  used  on  virtually  every  con¬ 
struction  project. 

System  Description 

DEFINITIONS: 

Lesson  Learned — systemic  problem  with  specifications  which 
caused  modifications  to  contracts. 

Project  Comment — A  umque  problem  with  a  specification  or  drawing 
which  may  cause  a  modification  to  a  contract. 

PURPOSE:  The  purpose  of  this  process  is  to  constantly  improve  the 
specifications. 

PROCEDURE:  Lessons  learned  must  be  written  and  sent  to  the  lessons- 
leamed  coordinator  for  electronic  entry  into  system  to  database  on  LAN.  The 
lessons-leamed  coordinator  reviews  entry  and  determines  if  entry  is  a  true 
lesson  learned  or  if  the  problem  should  be  corrected  by  another  means  (comment 
to  project,  memorandum  to  Division,  Suggestion  program,  etc.).  If  entry  is  valid, 
coordinator  routes  the  specification  problem  to  respective  Engineering  Division 
technical  section  for  review  and  recommendation.  Technical  section  reviews 
specification  problem  and  determines  if  problem  is  Omaha  District  Only  or 
Corps  of  Engineers  (COE)  problem.  If  only  Omaha  District,  specification 
problem  is  corrected  on  SPECNET  (Local  type  Specintact)  and  coordinator  is 
notified  specification  has  been  corrected.  If  specification  problem  is  COE  wide,  a 
DD3078  is  prepared  and  sent  through  channels  to  correct  COE  specifications 
and  notifies  the  lessons-leamed  coordinator.  When  a  lesson  learned  is  corrected 
or  determined  not  valid  any  more,  the  lesson  learned  is  removed  from  the 
database. 

CRITICAL  FEATURES: 

1.  Maximum  access  and  simplicity  for  status  of  lesson  learned. 

2.  Database  must  grow  and  shrink  to  remain  effective. 

Evaluation  Criteria 


Evaluation  measures  are  not  used  due  to  the  costs  of  maintaining.  Effectiveness 
is  measwed  by  reducing  the  number  of  repetitive  modifications. 
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System  Critique 

Process  is  very  efficient  and  works  well  for  the  time  being.  Source  code  for  the 
database  is  very  old  and  is  having  some  problems  with  the  newer  operating 
systems. 

Future  Direction 

Database  needs  to  be  upgraded  with  a  better  search  engine  and  brought  up  to 
current  operating  level  (Windows  level).  Budget  is  tight  so  it  is  hoped  to  find  a 
similar  database  program  and  modify  it  slightly;  perhaps  “Reviewer  Assistant 
System.” 
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Mobile  District's  Lessons-Learned  Page 

by  E.  William  East 

Mobile  District’s  representative  could  not  attend  the  workshop.  The  District  has 
been  very  active  in  the  eirea  of  lessons  learned.  For  more  information  on 
Mobile’s  contribution,  a  link  to  their  work  is  provided  on  the  Design  Review 
Tools  Steering  Committee  homepage  at  http://www.cecer.army.mil/pl/ra/ 
committee. 
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Perspectives  on  Lessons-Learned  Systems  From  South  Atlantic  Division 

by  Johnny  M.  Baggette‘“ 

Lessons  Learned  (LL)  is  a  subject  dear  to  my  heart.  I  think  this  is  the  key  to 
improving  the  quality  of  our  designs.  The  first  thing  I  would  like  to  do  is  define 
LL  for  this  paper.  The  term  is  often  used  for  various  types  of  feedback.  For  my 
purposes,  it  is  feedback  on  the  quality  of  a  project  under  construction.  An 
example  would  be  the  discovery  during  construction  that  criteria  such  as  a  guide 
specification  is  out  of  date  and  caused  a  construction  change.  The  LL  would 
define  the  problem  and  suggest  a  solution.  The  LL  System  (LLS)  would  be  a 
repository  for  these  LLs  until  the  specification  is  corrected.  In  this  case, 
someone  would  fill  out  an  ENG  Form  3078  and  submit  it  to  USAGE  for 
correction  of  the  specification. 

The  key  to  a  useful  LLS  is  a  system  that  is  simple  to  use  and  is  kept  up  to  date. 
The  Mobile  District  is  using  such  a  system,  based  on  a  system  originally 
developed  by  the  Omaha  District.  They  have  taken  the  system  and  made  it 
available  through  a  Home  Page  on  the  Internet.  If  all  Districts  had  such  a 
simple  system,  and  a  central  Home  Page  were  developed  for  all  the  Corps,  we 
could  have  a  truly  simple  and  useful  LLS.  The  central  system  could  access  the 
individual  district  systems  in  a  transparent  fashion  to  the  user. 

To  continue  to  be  useful,  an  LLS  must  be  used  to  make  corrections  to  our 
criteria,  and  then  they  must  be  removed  from  the  LLS.  The  other  thing  that 
must  happen  is  continuous  input  of  new  LLs.  This  can  be  encouraged  by  making 
sure  the  people  that  put  LLs  into  the  system  get  feedback  when  corrections  are 
made.  I  see  this  as  a  key  to  making  the  system  work  in  the  long  term. 

If  the  Corps  develops  a  central  system,  it  must  be  simple  as  described  above  and 
must  not  add  a  burden  on  the  Districts.  The  Districts  will  want  to  use  a  simple 
system  that  offers  good  LLs  and  doesn't  cost  them  to  support  it,  except  for  the 
time  within  their  District  to  keep  their  individual  system  up  to  date.  This 
valuable  tool  should  be  developed  and  made  available  as  soon  as  possible  with 
future  expansion  imtil  edl  Districts  are  participants.  These  are  just  some 
thoughts  I  have  and  a  summary  of  my  experience  on  this  subject.  I  stand  ready 
to  assist  with  the  development  of  such  a  system  in  whatever  capacity  is 
appropriate. 


Commander,  U.S.  Army  Engineer  Division,  South  Atlantic,  77  Forsyth  Street,  SW,  ATTN:  Mr.  Johnny  Baggette 
(CESAD-ET-EE),  Atianta,  GA  30335-6801 . 
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BCO  Reviews  at  Portland  District 

by  Joseph  B.  Russell” 

Organizational  Context 

The  number  of  personnel  involved  in  the  BCOE  process  within  the  District  is 
approximately  225  personnel.  This  number  includes  Engineering  Division, 
Operations  Division,  and  Construction  Division. 

Ineffectiveness  was  commonplace  with  old  methods  due  primarily  to  the  hard¬ 
copy  format  that  the  District  was  using.  Inefficiencies  were  due  in  part  to  the 
imcertainty  of  the  status  of  follow-up  actions.  The  designers  contacted  the  indi¬ 
vidual  reviewer  independently  to  discuss  the  comments.  This  made  it  hard  to 
identify  whether  comments  had  been  incorporated  within  the  final  bid  docu¬ 
ments.  Field  office  concerns  were  left  unresolved  in  many  cases.  This  created 
frustration  and  discontent  with  the  process. 

System  Description 

Portland  District  is  using  Microsoft  Office  Software  for  forms  assembly  and 
CCMail  for  electronic  mail  messaging  and  document  transfers  for  the  BCOE 
Review  Process. 

Evaluation  Criteria 

Management  measures  used  to  evaluate  the  successfulness  of  the  previous  and 
current  systems  are  imknown.  At  the  present  time  the  efforts  have  been  bottom 
driven  based  on  concerns  from  the  field  offices  and  people  who  have  moved  from 
the  field  office  back  into  Construction  Services  Branch  to  initiate  some  revisions. 

.  System  Critique 

The  electronic  form  of  the  NPD  Form  32  was  developed  by  a  Construction  Divi¬ 
sion  POC  at  first  in  Microsoft  Word.  The  BCOE  comments  are  transmitted  from 
field  offices  by  electronic  mail  as  an  attachment  file.  Files  are  imported  into  a 
master  document  and  critiqued  by  the  Construction  Division  POC  prior  to 
forwarding  to  the  Engineering  Technical  Manager.  The  ability  to  sort  comments 
based  on  spec  sections  or  drawing  reference  numbers  is  nearly  nonexistent  in 
Word.  Microsoft  Excel®  is  now  being  considered  to  accomplish  this  since  it  does 
have  a  sorting  capability. 


"  Commander,  U.S.  Army  Engineer  District,  Portland,  P.O.  Box  2946,  ATTN:  Mr.  Joseph  B.  Russell  (CENPP-CO- 
CS),  Portland,  OR  97229-2946. 
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Future  Directions 

A  more  efficient  way  to  store  and  retrieve  this  information  is  obviously  a  data¬ 
base  format.  This  is  true  since  specific  parameters  for  importing,  sorting,  edit¬ 
ing,  exporting,  storing,  and  retrieving  can  be  utilized  to  maximize  the  efficiency. 
We  are  pursuing  development  of  a  database  system  which  replicates  the  NPD 
Form  32  at  the  input  screen. 
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5  Products  To  Support  Design  Reviewers 
and  ARMS 


The  Reviewer's  Assistant  and  Lessons-Learned  Generator 

by  E.  William  East  and  Michael  C. 

Design  reviews  attempt  to  discover  and  eliminate  potential  problems  that  may 
be  encoimtered  during  the  construction  and  operation  of  a  facility.  The  charac¬ 
teristics  of  many  projects  suggest  that  design  reviews  are  necessary  for  ensuring 
a  balance  among  the  various,  conflicting  requirements  of  many  projects 
[O’Connor  1986].  Some  of  the  many  aspects  that  need  to  be  considered  during  a 
design  review  include:  (1)  biddability,  (2)  constructibility,  (3)  operability,  (4) 
technical  reviews,  and  (5)  customer  reviews. 

The  biddability  component  of  a  design  review  attempts  to  determine  “the  ease 
with  which  the  contract  docvunents  can  be  vmderstood,  bid,  administered  and 
enforced”  [Kirby  1988].  Constructibility  effects  refer  to  the  “compatibility  of  the 
design  with  the  site,  methods,  materials,  and  schedules”  [Kirby  1988].  Con¬ 
structibility  problems  may  result  in  “schedule  delays,  safety  detriment, 
structural  deficiencies,  additional  labor  costs  and  contract  disputes”  [Hancher 
and  Lutz  1988].  Operability  reviews  may  find  items  that,  if  not  resolved,  could 
residt  in  “facility  disruption,  diminished  habitability,  excessive  maintenance, 
safety  detriment,  energy  inefficiency,  structural  deficiency,  and  system  replace¬ 
ment”  [Hancher  and  Lutz  1988].  Specific  technical  reviews  are  also  conducted  to 
ensure  compliance  with  initial  and  design  calculations,  building  codes  and  inter¬ 
faces  between  disciplines  [USACOE  1987].  Customer  reviews  may  be  conducted 
to  ensure  that  the  user’s  functional  requirements  are  included  in  a  project. 
Other  examples  of  design  review  types  include  historical  preservation  reviews 
and  environmental  sustainability  reviews. 

Given  the  complexity  and  geographic  distances  involved  in  efficiently  conducting 
design  reviews,  the  U.S.  Army  Corps  of  Engineers  has  developed  an  automated 
system  for  the  support  of  the  design  review  process  called  the  Automated  Review 
Management  System  (ARMS)  [Kirby  1987].  The  process  is  shown  in  Figure  1. 
Project  managers  request  that  reviews  be  conducted  based  on  design  submittals 
received  from  Architect/Engineer  (A/E)  firms.  Review  managers  identify  specific 
reviewers  who  then  conduct  reviews  and  transmit  their  comments  to  the  A/E 
firm  for  resolution.  The  ARMS  system  is  reqxiired  for  use  on  the  U.S.  Army, 
Corps  of  Engineers,  Military  Construction  Program  Projects  [HQUSACE  1993]. 


“  Commander,  U.S.  Army  Construction  Engineering  Research  Laboratories,  P.O.  Box  9005,  ATTN:  CECER-PL-E, 
Champaign,  IL  61826-9005. 
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Figure  1.  Model  of  review  comment  preparation. 


Data  in  ARMS  are  stored  in  an  Oracle  database  operating  on  mini-computers 
running  the  UNDC  operating  system.  There  are  DOS  and  Windows  shells  for 
typing  comments  into  the  standard  ARMS  uploading  format.  The  uploading 
format  is  an  ASCII  file  with  a  combination  of  fixed  field  and  period  deUmited 
data  elements.  The  format  is  often  referred  to  as  a  “CMT”  file  since  the  file 
name  extension  of  “cmt”  is  used  for  a  correctly  formatted  ASCII  file.  Other  sys¬ 
tems  supporting  ARMS  may  also  use  the  CMT  format  to  transmit  data  to 
ARMS. 

The  Reviewer’s  Assistant  is  a  system  developed  at  the  U.S.  Army  Construction 
Engineering  Research  Laboratories  (USACERL)  to  assist  the  design  reviewer  in 
capturing  and  appropriately  reusing  their  design  review  experience  [East 
1995a].  Copies  of  the  system  have  been  made  available  to  over  500  govern¬ 
mental  organizations  and  private  companies  for  testing  [ASCE  1995;  McGraw- 
Hill  1996;  NASA  1996]. 

Reviewers  prepared  design  review  comments  used  as  the  basis  for  the  Review¬ 
er's  Assistant  System.  A  reviewer  critiques  plans  and  specifications  according  to 
the  reviewer's  expertise,  the  experience  of  others,  and  the  use  of  appropriate 
reference  tools.  The  reviewer's  mental  evaluation  of  the  plans  and  specifications 
is  translated  into  a  set  of  design  review  comments.  This  set  of  comments  is  then 
transmitted  either  electronically  to  the  ARMS  system  or  printed  and  mailed  to 
the  appropriate  party. 

As  the  Reviewer's  Assistant  is  used,  there  will  be  an  increasing  number  of 
projects  stored  in  the  system's  database.  During  one  fiscal  year,  for  example, 
several  himdred  projects  may  be  reviewed  by  a  given  reviewer.  Searching 
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through  a  potentially  large  set  of  comments  will,  over  time,  become  cumbersome 
for  the  user  and  slow  due  to  the  increasing  size  of  the  database.  A  method  is 
needed  for  easily  accessing  sets  of  highly  useful  review  comments,  so  that 
reviewers  may  efficiently  find  reusable  comments. 

Making  sense  of  the  large  and  ever  increasing  amoimt  of  information  contained 
in  databases  is  often  referred  to  as  “data  mining.”  The  interest  in  integrating 
the  ability  to  discover  patterns  in  large  databases,  a  subset  of  machine-learning 
technology,  is  expected  to  increase  as  organizations  find  themselves  with  ever 
larger  amounts  of  data  and  less  time  to  evaluate  that  data  [Michie  1990]. 
Interest  is  also  increasing  in  the  area  of  knowledge  discovery  in  databases  by 
many  researchers  [Silberschatz  1990]. 

Owners  have  attempted  to  transmit  lessons  learned  from  construction  to  design 
through  the  use  of  stemdalone  paper  or  automated  checklists.  Due  to  the  limited 
time  available  to  complete  the  hundreds  of  design  reviews  conducted  each  year, 
reviewers  can  not  practically  spend  the  time  needed  to  access  and  maintain 
these  standalone  systems.  In  the  author's  opinion,  for  lessons  learned  to  be  fully 
applied,  they  must  be  available  as  reviewers  compose  their  design  review 
comments.  The  authors  were  interested  in  exploring  the  extent  to  which  lessons 
learned  could  be  abstracted  directly  from  a  practical  comment  composition 
system  such  as  the  Reviewer’s  Assistant. 

The  following  section  of  this  paper  is  a  brief  overview  of  the  Reviewer’s 
Assistant.  A  complete  discussion  of  the  system  may  be  foimd  in  East  [1995b]. 
The  next. section  discusses  the  issue  of  quality  in  reviews  and  propose  a  simple 
algorithm  for  discovering  comments  of  high  quality.  The  third  section  contains  a 
run  of  the  Lessons-Leamed  Generator  on  a  sample  database.  The  results  of  the 
sample  run  demonstrate  problems  with  the  current  approach  that  are  discussed 
in  the  final  section  of  the  paper. 

The  Reviewer's  Assis^nt 

The  Reviewer’s  Assistant  integrates  a  relational  database  system,  a  text  editor, 
and  communications  software  into  a  seamless  sequence  of  reviewer-computer 
interactions  designed  to  support  the  design  review  process.  The  Reviewer’s 
Assistant  contains  two  models  of  user  interaction.  In  the  “novice  mode,”  which  is 
the  default  start-up  mode,  users  are  guided  through  a  series  of  simple  steps  that 
allow  the  most  efficient  creation  and  distribution  of  design  review  comments.  As 
users  become  more  familiar  ivith  the  program,  however,  they  may  want  to  follow 
a  different  set  of  steps  than  is  in  the  “novice  mode.”  In  the  “expert  mode,”  more 
experienced  users  may  select  operations  to  be  performed  based  on  their 
underst^ding  of  the  program  and  the  task  at  hand.  Access  to  data  in  the 
“expert  mode”  is  available  through  “pull-down”  menus. 

The  “novice  mode”  is  huilt  upon  the  premise  that  reviewer's  primary  interest  is 
in  completing  the  design  review,  and  not  in  learning  new  software  to  help  them 
perform  design  reviews.  As  a  result,  the  “novice  mode”  follows  the  steps  that 
reviewers  typically  take  when  performing  design  reviews.  Specifically,  the 
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“novice  mode”  user  interaction  follow  these  steps:  (1)  the  reviewer  selects  their 
name  from  the  set  of  all  reviewers  known  to  the  system,  (2)  a  new  review  project 
or  an  existing  review  project  is  selected,  (3)  a  brief  set  of  general  information 
about  that  project  is  collected  for  new  projects  or  confirmed  for  existing  projects, 
(4)  users  may  begin  creating  comments,  (5)  during  the  comment  authoring 
process,  the  reviewer  may  “pop-out”  to  a  search  mode  to  find  past  comments 
related  to  specific  topics  of  interest,  (6)  searched  comments  that  are  edited  will 
be  included  in  the  current  design  review,  unedited  comments  are  discarded  as 
not  applicable,  and  (7)  a  review  is  completed  and  comments  are  forwarded  to  a 
printer  or  via  modem  to  the  ARMS  system  using  a  CMT  file  [East  1994]. 

One  important  concern  that  was  raised  and  addressed  during  the  development 
of  the  search  routines  is  that  of  copying  standard  sets  of  comments  to  all  projects 
— ^the  electronic  equivalent  of  using  photocopying  equipment  to  create  a  design 
review.  In  the  Reviewer’s  Assistant,  comments  identified  by  a  search  are  not 
automatically  applied  to  a  given  design  review.  Users  must  select  those  com¬ 
ments  by  individually  editing  each  comment  that  applies  to  the  project.  The 
user  need  not  change  the  comment  text,  however,  the  mandatory  indices,  for 
specification  section  and  design  discipline,  must  contain  values.  Those  com¬ 
ments  that  are  not  edited  may  be  “dumped”  from  the  list  at  any  time.  Any 
remaining  imedited  comments  are  automatically  removed  prior  to  printing  the 
list  of  comments  for  the  project. 

Within  its  relational  database,  the  Reviewer’s  Assistant  maintains  each  com¬ 
ment  for  every  review.  Each  comment  is  uniquely  identified  with  a  combination 
of  project  nvimber,  comment  number,  and  comment  text  note  number.  Each 
comment  is  linked  with  indexing  information  corresponding  to  the  content  of  the 
comment  and  its  context  in  the  design  review  process  contained  in  other  tables. 
Searching  using  these  indices  allows  reviewers  to  easily  apply  expertise  from 
past  projects. 

The  most  important  of  the  indices  that  describe  comment  content  are  “Feature/ 
Value”  combinations.  The  Feature/Value  combinations  initially  provided  with 
the  system  are  organized  according  to  the  standard  Construction  Specification 
Institute  (CSI)  breakdown  [CSI  1988].  Figure  2  shows  an  example  of  the  tree 
structure  used  to  display  the  Feature/Value  pairs.  The  Reviewer’s  Assistant 
initially  comes  with  the  top-level  Feature  set  to  “Specification  Sections,”  and 
Values  set  to  the  specification  paragraphs  imder  the  specification  section.  The 
combination  of  Featimes  and  Values  describes  the  kind(s)  of  work  to  which  the 
review  comment  is  related. 

Dviring  the  initial  set-up  of  the  program,  the  Feature/V alue  tree  may  be  modified 
to  reflect  any  type  of  project.  Nested  sets  of  Features  may  be  created  to  model 
complex  types  of  indices.  Each  of  the  Features  may  contain  associated  Values; 
however,  the  typical  situation  is  that  the  Values  are  at  the  lowest  level  of  a  set  of 
nested  Features. 
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Revieuer  s  ^^ssistant  v.0.95 

File  Search  Edit 

* — ^Spec  Sections 

— General  Requirements 
— ^Siteuork 
— Concrete 
— Masonry 
— Metals 

— Wood  and  Plastics 
— Iliernal  &  Moisture  Protect i 
— ^Doors  and  Uindows 
— ^Finishes 
— ^Specialties 
— ^Equipment 
— ^Furnisliings 
— Special  Construction 
— Conueyinq  Systems 
— Mechanical 
— ^Electrical 

— Contracts  and  Conditions 


Project:  Roof  Repair  Building  T-ZOO 

Print  System 


Waterproofing 
Danpproof ing 
Water  Repellents 
Mapor  Retarders 
Air  Barriers 
Insulation 

Ext  Insulation  &  Finish  System 

Fireproofing 

Firestopping 

Shingles  and  Roofing  Tiles 
Manufactured  Roofing  d  Siding 
Exterior  Wall  Assemblies 
►Membrane  Roofing 
traffic  Coatings 
Flashing  and  Sheet  Metal 
Roof  Specialties  d  Accessories 
Skylights 
Joint  Sealers 

THERM,  d  MOIST.  PRO.  (GENERAL) 


mmmm 


Figure  2.  Feature/value  pairs. 


The  edit  screen,  shown  in  Figure  3,  is  where  conunents  are  created  and  indices 
are  linked  to  each  comment  instance.  The  text  in  the  center  of  Figure  4  is  the 
instance  of  the  comment  currently  being  edited.  The  Function  key  [F3]  allows 
users  to  edit  the  project  specific  location  of  the  comment.  Function  key  [F4] 
allows  users  to  edit  the  FeatureA^alue  pairs  for  the  comment.  The  Fimction 
keys  [F5]-[F8]  allow  users  to  select  Perspectives  for  the  comment.  Pressing  [F9] 
allows  users  to  modify  the  set  of  keywords  associated  with  the  current  comment. 

In  Figure  3,  the  Feature  selected  is  “Thermal  and  Moisture  Protection,”  and  its 
Values  are  “Waterproofing,”  “Dampproofing,”  “Water  Repellents,”  “Vapor 
Retarders,”  etc.  Since  the  content  of  the  comment  may  refer  to  a  number  of  dif¬ 
ferent  Specification  Sections,  the  Reviewer's  Assistant  allows  any  number  of 
FeatureA^ alue  combinations  to  be  linked  to  the  comment. 

Another  form  of  indexing  information  eire  the  “Perspectives.”  While  Feature/ 
Value  pairs  identify  comment  content,  Perspectives  identify  the  comment's 
context.  Four  types  of  Perspectives  are  initially  provided  in  the  Reviewer's 
Assistant:  “Design  Discipline,”  “Review  Type,”  “Site  Criteria,”  and  “Reviewer.” 
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Reuieuer  s  Assistant  u.0.95 

Search  Edit 


Project:  Roof  Repair  Building  T-ZQO 

Print  _ Systen 


[F3]  to  Edit: 


Spec  H 
Betaii  II 
Sheet  U 


07510 


[F4]=  Spec.:  Henbrane  Roofing 
[F5]=  ARHS  Disc,:  Architectural 

[F6]=  Reu ieuers:  Bill  East 

[F7]=  Reuiew  Type:  35Z  Concept  Revieu  + 

[F83=  Site  Criteria: 

tF91=  Keyuords:  Felt  Ply  Roof 

Stone 


Specifications  call  for  a  3  ply  built  up  roof  systen.  Houeuer 
because  the  top  felt  is  a  stone  enbedded  felt  a  three  ply  systen 
cannot  be  used.  Review  roof  systen  utilized  to  assure  satisfactory 
systen  is  in  place,  and  assure  that  error  is  not  repeated. 


Figure  3.  The  comment  edit  screen. 


As  shown  in  Figure  3,  Perspectives  are  identified  in  the  upper  righthand  portion 
of  the  comment  edit  screen  under  the  function  keys  [F5],  [F6],  [F7],  and  [F8] 
respectively.  The  Design  Discipline  Perspective  allows  the  identification  for  the 
field  of  expertise  of  the  firm  that  should  evaluate  the  comment.  Examples  of  De¬ 
sign  Disciplines  are  “Mechanical,”  “Structural,”  and  “Architectm-al.”  Design 
Discipline  could  also  be  used  as  address  information  by  including  the  name  of 
the  consultant  who  should  evaluate  the  item  in  question. 

The  Review  Type  Perspective  is  used  to  identify  the  timing  of  the  comment  in 
the  review  process.  This  is  important  because  the  specificity  of  comments  varies 
as  the  design  progresses.  At  the  start  of  a  design,  for  example,  comments  of  a 
more  general  nature  are  typically  encountered.  The  Review  Type  may  be  used  to 
distinguish  between  concept  and  final  design  reviews  to  allow  searching  on  the 
correct  level  of  detail. 

The  Site  Criteria  Perspective  may  be  used  to  identify  various  information  re¬ 
garding  the  overall  project,  for  example,  geographical  location  or  customer  name. 
The  Reviewer  Perspective  provides  an  index  for  the  name  of  the  person  con¬ 
ducting  the  review.  Although  the  set  of  Perspectives  is  initially  set  in  the 
Reviewer's  Assistant,  both  the  names  and  the  contents  of  all  of  the  Perspectives 
may  be  changed.  For  example,  the  Site  Criteria  Perspective  may  be  changed  to 
“Customer  Criteria”  and  used  to  track  items  of  interest  to  specific  customers. 

The  third  category  of  indexing  information  is  “Keywords.”  The  Reviewer's  Assis¬ 
tant  contains  over  2,000  construction  related  ke3words.  Programming  also 
identifies  the  plural  forms  of  these  keywords.  Reviewers  may  link  up  to  six 
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keywords  to  a  review  comment.  As  shown  in  Figure  3,  Keywords  are  linked  to 
each  comment  under  the  function  key  [F9]  in  the  upper  righthand  box  of  the 
comment  edit  screen. 

Discovering  and  Abstracting  High  Quaiity  Comments 

As  of  Fall  1992,  the  ARMS  “Central”  site  contained  over  three  million  design 
review  comments,  primarily  from  a  single  Corps  of  Engineers  District.  The 
ARMS  system  has  become  more  decentralized  since  1992  and  each  local  ARMS 
server  system  will  accumulate  comments  at  a  rapid  rate  as  more  reviews  are 
completed.  This  review  comment  population  explosion  will  create  severe  prob¬ 
lems  for  reviewers  performing  searches  for  previously  created  comments.  As  the 
number  of  comments  in  the  system  increases,  a  search  of  the  database  will  yield 
an  increasing  number  of  comments  of  varying  degrees  of  usefulness  to  the 
reviewer.  Without  some  notion  of  quality  to  order  the  retrieved  information  or 
to  constrain  the  search,  the  reviewer  is  forced  to  examine  all  of  the  retrieved 
comments  arbitrarily  to  find  the  comments  that  apply  to  the  particular  project 
at  hand. 

Defining  High  Quality  Comments 

Qiis^lity  in  review  comments  may  be  measured  by  three  metrics;  usefulness, 
generality,  and  content  stability.  Usefulness  refers  to  the  actual  content  of  the 
comment;  it  measures  how  well  a  problem  and  its  solution  are  described  and  the 
salience  of  the  problem/solution  to  the  design  review  process.  Generality  refers 
to  the  applicability  of  the  comment  across  projects.  Many  design  review  com¬ 
ments  are  specific  to  a  single  or  a  small  set  of  projects.  While  these  types  of 
comments  may  be  very  useful  to  their  parent  projects,  the  applicability  of  com¬ 
ments  that  are  very  project-specific  to  future  projects  is  often  limited.  Of  course, 
a  comment  cannot  be  so  general  as  to  lack  sufficient  context  to  describe  what 
must  be  done.  Below  are  several  examples  of  comments  that  are  not  well 
formed: 

The  design  team  shall  consult  with  the  Base  Civil  Engineering  Office  to 
review  the  installation's  program  of  architectural  compatibility.  The  design 
team  shall  be  sensitive  to  the  cultxiral,  architectural,  and  environmental 
influences  that  affect  the  installation  and  the  particular  site  proposed  for  the 
project. 

The  visual  design  of  the  project  should  be  in  harmony  with  its  surrounding. 

Secure  rooms  and  vaults  have  bars  in  ducts. 

Detail  C  on  sheet  A-7  does  not  clearly  show  how  or  if  the  cab  glazing  units  are 
anchored  at  the  jambs. 

An  example  of  a  comment  that  is  of  high  quality  is  provided  below.  The  example 
comment  explains  the  current  situation,  the  rationale  explaining  the  potential 
problem,  and  contains  a  possible  remedy  for  the  situation  based  on  the 
reviewer’s  knowledge. 
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The  specification  indicates  copper  roof  pan  lengths  to  be  approximately  45' 
long.  The  Copper  Development  Association  recommends  30'  maximum  pan 
lengths,  especially  in  northern  tier  climates.  Copper  expands  1/8”  per  10’  for 
every  100  °F  of  temperature  change.  The  45'  long  pans  with  expansion  cleats 
are  theoretically  possible,  but  not  practicable  during  installation. 

Content  stability  is  the  third  concern.  As  comments  are  copied  from  project  to 
project,  reviewers  may  alter  the  contents  of  the  comment,  thus  causing  “content 
shift.”  Comments  experiencing  severe  content  shift  cannot  be  abstracted,  since 
it  is  imclear  what  the  meaning  of  the  comment  has  become. 

Abstracting  High  Quality  Comments 

The  Lessons-Learned  Generator  finds  and  abstracts  high  quality  comments  by 
observing  the  patterns  of  comment  reuse  by  reviewers.  This  approach  is  valid 
for  two  reasons.  The  first  is  that  there  is  a  historical  precedent  for  comment 
reuse.  Paper  sets  of  repetitive  deficiency  lists  are  frequently  developed  in  all 
agencies.  The  idea  of  problems  recurring  and  needing  to  be  fixed  is  a  common 
theme  in  most  design  reviews.  The  second  is  simply  that  people  would  rather 
reuse  well-formed,  on-point  comments  that  are  “tried  and  true”  rather  than  start 
from  scratch  to  create  a  well-formed  comment. 

As  reviews  are  conducted,  those  comments  that  are  well-formed  and  on-point  for 
a  partictdar  situation  will  tend  to  be  selected  imder  similar  situations  in  the 
futm-e.  Those  comments  that  are  too  vague  or  too  general  will  not  be  used  again. 
High  quality  comments  are  those  comments  that  reviewers  search  for,  find 
useful  and  use  again  and  again  on  their  current  project.  Based  on  this  defi¬ 
nition,  we  can  assert  that  high  quality  conunents  will  have  more  than  a  singular, 
or  infrequent,  existence  in  the  comment  database. 

The  Lessons-Learned  Generator  discovers  and  represents  these  patterns  of  com¬ 
ment  usage  in  a  two-phase  process.  First,  a  comment  frequency  threshold  is 
calculated  from  the  existing  Reviewer's  Assistant  database.  Comments  whose 
frequency  exceed  this  threshold  are  hypothesized  as  high  quality  comments. 
The  second  phase  involves  a  closer  examination  of  those  hypothesized  com¬ 
ments.  The  content  of  the  comment  is  analyzed  for  commonalties  using  the 
indexing  information  linlted  to  every  instance  of  the  comment.  If  there  are  suffi¬ 
cient  commonalities  in  the  indexing  information,  then  it  is  assumed  that  content 
shift  has  not  occurred,  and  the  comment  is  abstracted  to  a  new  project  called 
“Lessons-Learned  Comments.”  The  Lessons-Learned  Generator  has  several 
heuristics  to  judge  whether  a  comment  has  experienced  content  shift. 

The  Lessons-Learned  Generator  Algorithm 

To  illustrate  the  algorithm,  consider  a  sample  database  containing  five  projects 
emd  five  comments  shown  below.  When  we  look  at  the  set  of  comments  for  the 
five  projects,  the  first  three  comments  have  only  been  used  once  on  individual 
projects.  The  fourth  comment  was  created  during  the  review  of  the  first  project 
and  subsequently  copied  to  each  of  the  other  foiir  projects.  The  last  comment  in 
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the  example  database  was  created  on  a  project  and  applied  to  two  subsequent 
projects.  In  the  Reviewer's  Assistant,  information  of  this  type  may  be  drawn 
from  the  relational  database  structure  since  comment  text  is  identified  by  both  a 
comment  number  and  an  instance  number. 


Comment  Number 

Number  of  Instances 

1 

1 

2 

1 

3 

1 

4 

5 

5 

3 

Phase  1:  Calculating  a  comment  frequency  threshold 


The  lessons-leamed  generator  begins  by  determining  a  “frequency  threshold”  to 
compare  the  amoxmt  of  reuse  in  the  database  that  is  being  evaluated  A  table  of 
comment  frequencies,  with  each  entry  in  the  table  corresponding  to  the  number 
of  times  a  particular  comment  is  developed  for  every  comment  in  the  database. 
This  is  done  by  coimting  the  number  of  comment  text  records  (“notes”)  asso¬ 
ciated  with  each  unique  comment  identification  number. 

Next,  the  expected  comment  Ifrequency  is  calculated.  This  average  comment 
frequency  is  the  threshold  that  will  be  used  to  evaluate  frequently  reused 
comments.  For  the  above  example,  the  comment  frequency  threshold  value  is: 
(1/5)  *(1  +  1  +  1-h5-h3)  =  (1/5)  *  11  =  2.2.  Comments  Four  and  Five,  whose  fre¬ 
quencies  exceed  2.2,  are  evaluated  in  the  abstraction  phase. 

Phase  2:  Abstracting  the  Selected  Comments 

For  each  comment  whose  frequency  is  greater  than  the  threshold,  all  instances 
of  the  comment  are  found  and  the  associated  index  information  is  evaluated. 
The  objective  of  gathering  this  information  is  to  determine  the  commonalties,  or 
“conditional  attributes,”  for  the  comment  [Zairko  1991].  In  our  sample  database, 
Conunent  Five  has  three  instances;  suppose  that  the  instances  have  the  Feature/ 
Value  pairs  associated  with  them  as  shown  below. 

Instance  1: 

Thermal  and  Moisture  Protection/W aterproofing 
Thermal  and  Moisture  Protection/Water  Repellents 

Instance  2: 

Thermal  and  Moisture  Protection/Waterproofing 
Doors  and  Windows/Metal  Doors  and  Windows 

Instance  3: 

Thermal  and  Moisture  Protection/Waterproofing 
Thermal  and  Moisture  Protection/Dampproofing 
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The  first  instance  of  Comment  Five,  for  example,  has  two  FeatureA^alue  links. 
The  first  is  to  the  Thermal  and  Moisture  Protection  Featiire  with  a  value  of 
Waterproofing.  The  first  instance  of  Comment  Five  also  is  linked  to  another 
FeatureA^alue  pair.  Thermal  and  Moisture  ProtectionAVater  Repellents. 

The  intersection  of  the  FeatureA^alue  combinations  of  the  three  sets  is  com¬ 
puted.  In  this  case,  only  a  single  FeatureA^alue  combination  exists  in  all  three 
instances:  Thermal  and  Moisture  Protection/Waterproofing.  This  particular 
Feature/Value  combination  becomes  a  conditional  attribute  for  Comment  Five. 
An  identical  operation  is  performed  on  the  perspectives  and  keywords. 

Both  Ke5rwords  and  FeatureA^alues  are  critical  conditional  attributes  in  deter¬ 
mining  the  content  of  a  comment.  The  comment's  instances  must  have  at  least 
one  element  both  in  the  intersection  of  the  instances'  keyword  sets  and  in  the 
intersection  of  the  instemces'  FeatxireA^alue  combination  sets.  The  algorithm 
interprets  a  null  intersection  set  for  either  t)T)e  of  index  information  as  a  set  of 
comments  that  have  undergone  some  type  of  content  shift.  As  a  result  of  this 
shift  the  specific  instances  of  the  comment  are  not  abstractable  to  a  single 
comment. 

An  abstracted  comment  is  a  combination  of  the  comment  text  and  the  condition 
attributes  created  by  the  index  information  analysis.  Each  abstracted  comment 
is  saved  in  a  new  project  within  the  Reviewer’s  Assistant  database.  All  other 
instances  of  the  comment  are  deleted.  Reviewers  searching  the  lessons-leamed 
database  can  access  the  information  that  was  retained  most  frequently  before 
having  tp  sift  through  the  results  of  a  full,  unordered  search  through  the 
projects.  The  lessons-leamed  database  may  also  serve  as  a  back-check  for  design 
reviewers  and  as  a  learning  tool  for  novice  review  personnel. 

A  Sample  Run  of  the  Lessons-Leamed  Generator 

To  illustrate  the  Lessons-Leamed  Generator,  we  created  a  small  test  bed  con¬ 
sisting  of  five  reviews  in  the  domain  of  roofing  systems.  The  reviews  were  dis¬ 
tributed  across  copper  roofs  and  shingle-asphalt  roofs.  For  each  project, 
searches  obtained  comments  from  previous  projects,  and  new  comments  pertain¬ 
ing  to  the  specific  design  were  added. 

Figure  4  is  the  screen  that  the  Lessons-Leamed  Generator  displays  after  the  fre¬ 
quency  threshold  calcvilation  phase.  The  comment  frequencies  have  been  tabu¬ 
lated  and  the  frequency  threshold  has  been  computed. 

After  the  abstraction  process,  the  Lessons-Learaed  Generator  displays  the  final 
screen  of  statistics.  Figure  5  shows  that,  of  the  18  comments  thought  to  be 
significant,  six  were  actually  extracted.  The  comments  that  were  not  abstracted 
exhibited  comment  shift. 
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Figure  4.  Results  of  the  frequency  threshold  evaluation  phase. 

Recommended  Improvements 

The  Reviewer's  Assistant  is  currently  being  tested  by  members  of  design,  con¬ 
struction,  and  owner  communities.  As  a  result,  large  databases  of  real  reviews 
created  over  a  period  of  time  do  not  yet  exist.  As  these  databases  become 
available,  the  authors  will  be  better  able  to  assess  the  performance  of  the 
Lessons-Leamed  Generator.  However,  even  the  small  test  discussed  in  this 
paper  demonstrates  future  areas  of  improvement  and  research. 

One  area  of  improvement  is  that  of  the  abstraction  heuristics,  specifically  the 
deciding  of  the  conditional  attributes  and  the  determination  of  content  shift. 
The  current  algorithm  assumes  that,  as  reviewers  modify  their  comments,  they 
modify  the  index  information  linked  to  the  comment.  As  this  may  not  always  be 
the  case,  it  would  be  useful  to  enforce  the  re-examination  of  the  index  infor¬ 
mation  when  content  shift  is  detected.  A  good  detection  measure  would  include 
analysis  of  the  index  information  that  was  changed,  and  perhaps  an  examina¬ 
tion  of  the  text  of  the  comment  itself.  Another  means  would  be  to  conduct  a 
keyword  analysis  of  each  comment  and  determine  if  the  order  of  all  keywords  in 
the  comment  was  consistent  with  the  instance  of  the  comment  originally  copied 
to  the  project  following  a  search. 

Allowing  the  program  to  check  keywords  just  prior  to  exiting  the  comment  edit 
screen  would  require  a  very  small  modification  to  the  Reviewer's  Assistant 
system.  The  change  could  also  be  performed  so  that  the  user  would  not  be  bur¬ 
dened  with  the  burdensome  overhead  of  manually  verifying  comment’s  indices. 
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Figure  5.  Results  of  the  extraction  phase. 


A  maximum  of  two  programmer  weeks  would  be  required  to  implement  such  a 
change. 

The  cmrent  method  used  to  calculate  the  frequency  threshold  may  be  fooled  by 
unexpected  comment  frequency  distributions.  For  example,  if  there  are  only  a 
few  unique  comments  in  the  database  when  the  Lessons-Leamed  Generator  is 
run,  a  small  group  of  comments  Avith  very  high  frequencies  may  push  the  fre¬ 
quency  threshold  beyond  the  frequencies  of  other  important  comments.  As  a 
result,  only  the  comments  from  the  very  high  frequency  group  are  examined  for 
abstraction  by  the  generator  while  equally  good  “lessons  learned”  may  not  be 
considered. 

Another  situation  that  causes  the  current  algorithm  to  act  in  an  unexpected  way 
is  the  case  where  a  comment  has  exceeded  the  frequency  threshold,  and  there  is 
not  a  single  common  conditional  attribute  across  all  instances.  For  example,  a 
comment  with  51  instances  where  the  last  instance  has  exactly  the  same  text 
but  all  the  conditional  attributes  have  been  changed.  Since  the  lessons-leamed 
algorithm,  which  is  searching  for  the  set  of  attributes  that  are  in  common  among 
all  instances  of  a  comment,  finds  not  a  single  common  attribute,  the  candidate 
comment  will  be  eliminated  from  further  consideration. 

Having  one  comment  with  a  completely  different  set  of  attributes  out  of  a  large 
set  of  similar  comments  is  unlikely,  since  users  will  naturally  limit  changes  to 
comment  indices,  which  the  user  perceives  as  extra  work.  Currently,  there  are 
two  approaches  to  solving  this  problem.  The  first  is  to  use  keywords  which,  as 
noted  previously,  is  a  more  accxuate  method  of  determining  if  attributes  are 
changed.  The  second  method  wotdd  be  to  develop  a  “noise”  threshold.  This 
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threshold  would  allow  a  few  pieces  of  inconsistent  data  to  be  removed  from  the 
evaluation  process. 

Two  approaches  to  setting  a  noise  threshold  have  been  suggested.  An  empirical 
method  would  ignore  comments  with  dissimilar  attributes  if  the  number  of 
comments  was  less  than  a  statistically  or  user-provided  threshold.  Another 
approach  that  could  be  implemented  would  eliminate  noisy  comments  after 
those  comments  have  remained  in  the  database  for  longer  than  a  user-defined 
duration. 

To  evaluate  the  possibility  of  unexpected  frequency  distributions,  detailed  analy¬ 
sis  of  large  databases  containing  design  review  comments  will  be  needed.  One 
author  has  suggested  that  a  database  of  approximately  1,000  items  is  required 
prior  to  effectively  testing  an  algorithm  of  this  type  [Frawley  1991].  Individuals 
using  the  Reviewer's  Assistant  are  asked  to  contact  the  authors  so  that  large 
databases  of  comments  may  be  tested. 

Some  have  suggested  that  a  possible  side  benefit  of  the  Lessons-Leamed  Gener¬ 
ator,  that  has  not  been  fully  explored,  may  be  in  the  use  of  comment  frequency 
information  and  the  comment  creation  date  to  cull  obsolete  comments  from  the 
database.  Currently,  the  recommended  way  to  remove  unneeded  comments  is  to 
delete  entire  projects  or  to  delete  individual  comment  instances  after  saving 
projects  in  CMT  files  or  backing  up  the  entire  database. 

While  the  Lessons-Leamed  System  may  be  considered  a  simple  type  of 
“imsuperyised”  learning  system,  some  authors  have  suggested  that  user  inter¬ 
action  would  improve  the  performance  of  this  type  of  algorithm.  For  example, 
frequency  thresholds  may  be  fine  tuned  with  user  interaction  or  subsequent 
analysis  [Cai  et  al.  1991], 

Further  evaluation  of  the  conditional  attributes  should  also  be  considered.  For 
example,  if  all  instances  of  a  comment  have  the  same  attributes,  a  possible 
generalizing  attribute  may  be  created  [Silverman  1991].  Conversely,  conditional 
attributes  may  be  specialized  to  only  a  very  small  subset  of  indices.  These 
attributes  may  even  be  considered  as  antecedents  of  rule  sets  that  determine 
when  specific  comments  may  be  applied  through  an  expert-system  mode  of 
interaction. 

In  developing  the  Lessons-Leamed  Generator,  the  authors  were  limited  in  their 
abilities  to  represent  design  review  comments  by  the  indices  included  in  the 
Reviewer's  Assistant  system.  Since  all  representations  are  limited,  the  authors' 
interest  was  not  to  develop  an  enhanced  representation  of  lessons  learned  but  to 
investigate  the  extraction  of  useful  lessons  learned  from  a  practical  relational 
database  system.  Application  of  “data  mining”  techniques  could  surely  be 
improved  if  the  representation  of  the  design  review  comments  was  extended 
beyond  the  simple  relational  model. 
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Conclusion 

The  Lessons-Leamed  Generator  is  a  demonstration  program  that  works  in  the 
context  of  the  Reviewer's  Assistant  system  to  abstract  a  set  of  high  quality  com¬ 
ments  from  a  large  set  of  reviews.  Quality  is  defined  in  terms  of  usefulness, 
generality/specificity,  and  content  stability.  A  high  quality  comment  is  one 
which  addresses  an  important  problem  and  is  clearly  and  concisely  written.  The 
algorithm  used  by  the  Generator  relies  on  patterns  of  reviewer  usage  to  deter¬ 
mine  the  quality  of  comments. 

Most  reviewers  do  not  have  time  to  access  standalone  repetitive  deficiency 
checklists.  Data-mining  tools,  operating  within  existing  corporate  automation 
systems,  allow  users  to  access  lessons  learned  during  the  course  of  normal  busi¬ 
ness  practices.  Use  of  these  lessons  learned  is  an  important  factor  in  reducing 
the  life-cycle  cost  of  construction. 
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Checking  Codes  and  Standards 

by  Carl  Mileff,  CMA  &  Associates 

Plan  review  involves  using  and  applying  a  large  amoxmt  of  information.  For 
example,  here  is  a  list  of  codes  and  standards  routinely  used  in  plan  check; 
building,  mechanical,  plumbing  and  electrical  codes;  fire  codes  and  related 
NFPA  Standards;  state  and  local  ordinances  and  code  amendments;  building 
and  industry  construction  and  material  standards;  engineering  and  structural 
references,  calculations;  code  interpretations,  publications,  personal  notes  and 
diagrams;  and  in-house  standard  forms  and  checklists. 

For  the  most  part,  this  data  is  in  paper  form,  voluminous,  and  very  difficult  to 
organize  and  correlate.  It  amounts  to  a  lot  of  information  to  be  searched,  ana¬ 
lyzed,  and  applied.  How  well  this  data  is  organized  and  made  available  has  a 
direct  impact  on  the  ability  of  reviewers  to  do  their  work  quickly  and  accurately. 

Plan  Review  Approach 

One  method  that  we  use  to  maintain  speed  and  accuracy  is  “combination 
review,”  where  each  plan  checker  reviews  and  manages  the  review  of  each  job 
from  start  to  finish,  covering  all  disciplines  in  one  session.  This  offers  benefits: 
faster  reviews,  continuity,  self-training,  versatility,  and  increasing  capabilities. 
It  has  its  problems  as  well.  Needed  information  has  to  be  quickly  available, 
usable,  and  in  easily  communicated  form. 

With  pressures  to  produce,  we  continue  to  experiment  with  different  text-based 
methods  to  perform  reviews  and  create  correction  reports:  standard  comments; 
checklists;  barcode  comments;  macros;  like-project  reports.  While  helpful,  these 
methods  prove  less  than  adequate.  In  every  case,  reviewers  spend  most  of  their 
time  editing  and  scrolling  through  unneeded  information  to  find  applicable  data 
and  create  a  report.  We  always  wanted  something  better. 

nancheck  Tools 

With  a  captive  programmer  (my  son),  we  started  the  process  of  designing 
PlanCheck  Tools  in  1986.  This  was  a  personal  experiment  for  my  company  that 
grew  into  a  real  product.  The  effort  lacked  many  of  the  essential  ingredients  for 
success,  but  it  was  a  valuable  experience. 

Simply,  I  wanted  to  be  able  to  review  a  plan,  turn  pages  with  my  left  hand,  and 
point  and  click  on  the  computer  with  my  right-handed  mouse.  I  wanted  to  view 
the  information  I  needed,  create  a  report,  print  it,  and  send  it  to  the  designers. 
It  would  have  the  following  basic  features:  repetitive  comments  could  be  stan¬ 
dardized,  saved  and  accessed;  comments  could  be  linked  to  codes  to  prevent 
arbitrary  findings;  we  could  store  and  link  all  kinds  of  information  (codes,  notes, 
pictures);  we  could  organize  information  for  different  kinds  of  buildings;  we 
could  create,  edit,  and  print  a  plan  check  report  automatically. 
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As  it  developed,  the  program  was  organized  into  a  hierarchy  of  topics  and 
internal  relationships:  Group  Search  Panel  by  building  types;  Main  &  Sub 
Topics  Reference  Codes — all  comments  have  five  references;  Comments — here 
comments  are  an  output  of  knowledge;  Notes  &  Diagrams;  Checklists;  Correc¬ 
tion  List  Files;  and  Printed  Reports. 

Users  could  configure  and  input  data  and  their  linkages.  Codes  and  other  data 
could  be  searched  by  subject  via  the  search  panel  or  by  keyword.  All  comments 
would  be  tied  to  reference  codes  to  prevent  arbitrary  comments.  A  plan  check 
file  could  be  created  and  include  project  data  and  the  added  correction  Ust 
comments.  There  would  be  editing  functions  and  a  printed  report. 

Filling  in  the  database  was  a  difficult  and  impossible  task  at  the  time.  The  code 
publishing  organizations  were  reluctant  to  share  their  copyrighted  material  and 
were  concerned  about  security.  We  received  a  limited  license  from  ICBO  to  copy 
small  portions  of  the  UBC.  The  bulk  of  any  necessary  data  had  to  be  input  by 
the  user.  This  was  a  major  drawback,  since  codes  were  essential  to  program 
operation. 

Lessons  Learned 

The  following  describes  some  of  the  lessons  learned  from  the  PlanCheck  Tools 
programming  project. 

Program  Design 

Initially,  the  program  was  written  vmder  Microsoft  Windows  Version  1.0.  This 
alone  presented  many  difficult  and  complex  technical  problems.  Documentation 
was  scarce  and  technical  support  virtually  nonexistent.  We  recognized  early  on 
that  most  users  would  not  be  familiar  with  Windows  and  would  have  to 
purchase  new  equipment  to  run  the  program.  This  resulted  in  users  rejecting 
the  system  based  on  hardware  costs  alone.  Also,  many  users  were  unfamiliar 
with  Windows  and  were  reluctant  to  retrain  themselves  or  their  staff.  Basic 
reference  code  data  and  configurations  were  input  into  the  program  during 
development  and  testing;  however,  the  very  limited  data  proved  inadequate  to 
cover  all  possibilities  and  additional  configuration  by  users  proved  too 
complicated  and  imworkable.  Though  it  was  the  best  we  could  come  up  with  at 
the  time,  the  linking  of  comments  to  reference  codes  was  a  mistake.  Comments 
were  the  final  product,  yet  you  could  not  get  to  them  except  via  the  applicable 
reference  code.  Session  files  were  another  problem.  Project  data  input  was  too 
cumbersome,  editing  too  difficult,  and  the  printed  list  was  poorly  structured  and 
not  of  word  processor  quality.  Output  was  one  of  the  most  critical  problems  and 
absolutely  prevented  practical  usability  of  the  program. 

Program  Development 

After  the  initial  design  phase,  many  features  continued  to  be  added,  mostly  as 
“good  ideas”  or  experiments.  Indeed,  features  took  precedence  over  usability  as 
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design  progressed.  This  prolonged  the  development  cycle  and  eventually 
weakened  the  basic  structm*e  of  the  program.  In  retrospect,  the  feature  list 
should  have  been  frozen  early  on. 

PlanCheck  Tools  was  primarily  designed  to  help  a  user  build  a  correction  list 
document.  We  also  believed  that  once  a  user  started  inputting  information  into 
the  system,  that  information  could  be  used  later  to  help  identify  common 
problems  or  indicate  problems  based  on  probability. 

Users,  however,  were  not  interested  in  the  long-range  plans  for  the  system  and 
were  confused  by  terms  they  were  unfamiliar  with.  They  needed  to  solve  specific 
problems,  and  PlanCheck  Tools  was  too  rich  with  “complicated”  features  and 
others  that  just  did  not  work  very  well. 

The  program  was  built  to  be  very  generic  and  user  configurable.  Again,  this 
type  of  design  means  that  the  user  has  to  do  a  lot  of  work.  They  were  turned  off 
by  the  prospect  of  too  much  configuration. 

One  of  the  biggest  marketing  obstacles  was  that  the  genersd  market  was  not 
computer  technical,  and  users  were  reluctant  to  work  differently  or  wanted  the 
system  to  do  everything  for  them.  One  valuable  lesson  learned  is  that  of  the  old 
cliche:  “A  chain  is  only  as  strong  as  its  weakest  link(s).”  Finally,  we  ceased 
development  in  1990  for  a  number  of  reasons.  We  hope  to  continue  the  effort  at 
a  futime  date. 


USACERL  CP-97/71 


55 


6  Future  Directions  for  Design  Review 
Systems 

An  Automated  Design  Review  Assistant 

by  Michael  C. 

The  Support  Environment  for  Design  And  Review  (SEDAR)  is  a  graphical  expert 
critiquing  system  for  use  by  designers  and  reviewers  of  flat  and  low-slope  roofs. 
SEDAR  provides  assistance  during  the  design  process  through  the  use  of  its 
critiquing  strategies  (error  prevention,  error  correction,  and  design  review)  and 
its  design  suggestion  strategy.  SEDAR  better  integrates  the  design-review  pro¬ 
cess  used  by  many  Architect/Engineer  (A/E)  offices  to  ensure  design  quality. 

SEDAR  helps  roof  designers  by  providing  critiques  and  suggestions  as  the 
design  of  the  roof  progresses  using  the  error  prevention,  error  correction,  and 
design  suggestion  strategies.  By  providing  feedback  as  design  decisions  are 
made,  errors  can  be  prevented  or  detected  early  in  the  design  process.  SEDAR 
helps  design  reviewers  by  checking  the  correctness  of  a  design  by  using  design 
codes  stored  in  its  knowledge  base  (the  design  review  strategy).  Since  the 
process  of  design  review  is  inherently  a  time-consuming  and  resource-con¬ 
strained  process,  SEDAR  will  help  reviewers  by  providing  consistent  and  com¬ 
prehensive  reviews  of  the  design  using  the  design  codes  within  its  knowledge 
base.  Use  of  SEDAR  in  the  existing  roof  design  process  will  help  to  reduce 
premature  roof  failures  that  are  caused  by  poor  quality  designs.  Roof  failures 
resulting  from  errors  and  misjudgments  in  design  constitute  a  serious  legal 
threat  to  architects,  contractors,  and  manufacturers  alike  (Griffin  1982)  and 
result  in  high  repair  and  maintenance  costs  to  building  owners. 

SEDAR  focuses  the  content  of  its  critiques  and  suggestions  through  the  use  of 
functional  decomposition  of  the  roof  design  task  called  the  Designer's  Task 
Model  (DTM).  The  DTM  was  created  from  observations  of  how  experienced  roof 
designers  decompose  the  roof  design  task  into  interdependent  subtasks 
associated  with  the  layout  of  functional  subsystems,  such  as  the  drainage 
system  or  the  walkway  system.  SEDAR’s  focusing  strategy  uses  the  DTM  to 
flexibly  track  the  progress  of  roof  designers  so  that  SEDAR  can  provide  the  most 
relevant  critiques  at  appropriate  times  in  the  design  process.  A  prototype 
version  of  SEDAR  has  been  implemented  for  personal  computers  running 
Microsoft®  Windows  using  Goldworks  III™,  a  LISP-based  expert  system  shell, 
and  AutoCAD™,  a  computer-assisted  design  (CAD)  tool.  The  results  of  an 
evaluation  of  the  system  were  that  users  had  favorable  reviews  of  the  system. 


”  Commander,  U.S.  Army  Construction  Engineering  Research  Laboratories,  P.O.  Box  9005,  ATTN:  Michael  C. 
Fu  (CECER-FL-E),  Champaign,  IL  61826-9005. 
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that  SEDAR  helped  reduce  the  number  of  design  errors,  and  that  the  functional 
decomposition  of  the  DTM  matched  the  users'  conception  of  the  roof  design  task. 

The  SEDAR  Architecture 

The  architecture  of  SEDAR  is  shown  in  Figure  1.  The  User  Interface  is  the  com¬ 
munication  medium  between  the  designer  and  SEDAR.  The  user  may  add, 
delete,  or  move  design  objects  (e.g.,  roof  drains,  airhandling  imits,  walkways, 
etc.),  examine  the  state  of  the  DTM,  view  the  existing  critiques  on  the  design, 
and  turn  any  of  the  critiquing  strategies  on  or  off.  User  actions  are  communi¬ 
cated  to  the  Critic  Management  Agent,  which  selects  a  critiquing  strategy  to 
apply  and  updates  the  shared  data  structures  on  the  Blackboard  (specifically, 
the  DTM  and  the  design  representation)  to  reflect  the  modification.  It  then  acti¬ 
vates  the  appropriate  Critic  Agents  (here  the  Flat/Low-Slope  Roof  Agent),  which 
perform  the  design  analysis  according  to  the  selected  critiquing  strategy,  and 
translates  their  results  into  graphical/textual  critiques.  The  critiques  are  then 
sent  back  to  the  User  Interface  for  display.  This  basic  operating  cycle  is  called 
the  iterative  critiquing  cycle. 

The  Designer’s  Task  Modei 

The  DTM  is  the  central  component  of  SEDAR  and  is  used  to  guide  automated 
support  for  both  designers  and  reviewers.  It  is  a  hierarchy  of  possible  design 
tasks  that  might  be  encountered  by  a  user  d;iring  a  roof  design.  Figure  2  shows 
a  portion  of  the  DTM;  the  task  at  the  left,  RoofLayout,  is  the  most  abstract  task. 
The  leaf  nodes  of  the  hierarchy,  for  example,  Drain-Layout,  WalkwayLayout, 
etc.,  represent  the  design  of  specific  functional  subsystems.  Part-of  links,  shown 
as  solid  lines  in  Figure  2,  describe  the  task-subtask  relationships.  Before-task 
links  are  precedence  relations  between  tasks  observed  from  human  expert 
behavior.  Interferes-with  links  represent  possible  interference  among  tasks. 


Figure  1.  The  SEDAR  architecture. 
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Figure  2.  An  activation  pattern  on  the  Designer’s  Task  Model. 


Only  the  interferes-with  links  related  to  the  Air-Handler-Layout  task  are  shown 
in  Figure  2.  For  example,  the  Air-Handler-Layout  and  Walkway-Layout  tasks 
are  related  by  an  interferes-with  link  because  walkways  should  not  overlap  air- 
conditioning  units.  Each  subtask  in  the  DTM  is  associated  with  a  set  of  design 
codes  in  the  Flat/Low-Slope  Roof  Agent  specifying  acceptable  placement  condi¬ 
tions. 

As  a  designer  works  on  the  roof  design,  the  DTM  is  used  to  track  the  designer's 
focus  of  attention.  Each  task  in  the  DTM  is  either  an  inactive,  active,  or  focus 
task.  The  set  of  all  task  states  in  the  DTM  forms  an  activation  pattern.  Focus 
tasks  represent  SEDAR's  interpretation  of  the  user's  cxurent  focus.  Each  task  is 
associated  with  a  set  of  design  objects;  when  a  new  object  is  added  to  the  design, 
all  tasks  associated  with  the  object  and  all  of  the  tasks'  ancestors  in  the  part-of 
hierarchy  are  focus  tasks.  In  Figure  3,  the  user's  selection  of  a  masonry  chimney 
object  causes  the  Chimney-Layout  task  and  its  ancestor,  the  Equipment-Layout 
task,  to  become  focus  tasks.  Active  tasks  are  related  to  the  focus  tasks  by  an 
interferes-with- relation,  are  subtasks  of  a  task  with  an  interferes-with  relation 
to  a  focus  task,  or  were  focus  tasks  previously.  They  represent  tasks  that  have 
been  and  should  be  considered  by  the  user.  Finally,  inactive  tasks  are  those  that 
have  not  been  addressed  yet  by  the  user.  During  the  critiquing  episode,  SEDAR 
uses  only  those  design  codes  that  are  linked  with  focus  and  active  tasks  so  that 
the  resulting  critiques  and  suggestions  are  relevant  to  the  user's  focus  of 
attention.  When  the  user  selects  the  chimney  object,  SEDAR  uses  the  error 
prevention  strategy  to  show  “off-limits”  areas  on  the  design  based  on  the  design 
codes  in  the  Flat/Low-Slope  Roof  Agent  (Figure  3).  If  the  user  disregards  the 
advice  and  places  the  masonry  chimney  too  close  to  an  existing  chimney,  SEDAR 
generates  a  critique  of  the  chimney  placement  (Figure  4).  Thus  the  DTM  is  used 
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in  a  flexible  manner  to  track  instead  of  to  constrain  the  user's  behavior,  and 
results  in  relevant  critiques  and  design  suggestions. 

A  design  reviewer  may  use  the  design  review  strategy  to  consistently  and  com¬ 
prehensively  check  subsystems  of  a  roof  design  according  to  the  design  codes  in 
the  Flat/Low-Slope  Roof  Agent.  The  reviewer  selects  a  roof  subsystem  from  a 
textual  representation  of  the  DTM,  and  the  design  review  strategy  checks  the 
corresponding  subsystem  in  the  roof  design. 

Prototype  Evaluation 

A  prototype  of  SEDAR  was  evaluated  in  two  experiments.  The  first  experiment 
was  a  system  usability  evaluation,  which  rated  the  performance  of  SEDAR  along 
various  usability  issues.  While  the  full  results  of  this  experiment  are  reported 
elsewhere  (Fu  1995),  one  outcome  of  this  experiment  was  an  informal  validation 
of  the  functional  decomposition  of  roof  subsystems  of  the  DTM.  The  second 
experiment  measured  the  prototype  system's  error  reduction  effectiveness,  and 
showed  that  SEDAR  is  able  to  reduce  both  the  total  number  of  errors  and  the 
classes  of  error  made  by  roof  designers. 

The  two  classes  of  errors  that  the  system  was  not  able  to  prevent  were  opti¬ 
mality  issues  regarding  object  placement;  although  the  placement  of  the  design 
object  satisfied  the  design  codes,  the  object  was  placed  in  a  “suboptimal” 
location.  Although  the  SEDAR  prototype  does  not  deal  with  the  optimality  of 
subsystem  design,  recognizing  and  advising  in  these  situations  was  expressed  as 
a  need  by  the  system  evaluators  for  future  development.  Additionally,  we  are 


Figure  3.  Results  of  the  Error  Prevention  Critiquing  Strategy. 
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Figure  4.  Enlarged  view  of  a  Graphical/Textual  Critique. 


looking  at  ways  of  critiquing  and  supporting  designers  throughout  the  design 
process,  from  early  conceptual  design  to  later  detailed  design  (e.g.,  Brown  and 
Chandrasekaran  1986). 

Conclusions 

Fiuictional  information  is  needed  for  a  variety  of  tasks  in  the  design  process. 
This  paper  describes  the  use  of  a  functional  task  decomposition,  the  Designer's 
Task  Model,  to  provide  both  flexibility  and  relevance  to  an  automated  design 
critiquing  system  called  SEDAE.  The  goal  of  SEDAR  is  to  provide  automated 
support  for  designers  and  reviewers  which  can  effectively  and  efficiently  reduce 
the  number  of  errors  made  in  the  design  process.  A  prototype  implementation  of 
SEDAR  has  been  evaluated,  and  showed  that  SEDAR's  critiquing  strategies 
successfully  reduced  the  number  of  errors  made  by  designers. 

References 

Brown,  D.,  and  B.  Chandrasekaren,  (1986).  “Knowledge  and  control  for  a  mechanical 
design  expert  system.”  IEEE  Computer,  19(7),  92-100. 

East,  E.,  et  al.,  (1995).  “The  Reviewer's  Assistant  System:  System  Design  Analysis  and 
Description.”  Technical  Report  FF-95/09/ADA294604,  USACERL,  Champaign,  IL. 

Fu,  M.,  (1995).  Using  A  Task-Based  Model  of  Design  for  Flexible  Control  in  an  Expert 
Critiquing  System,  Masters  thesis.  University  of  Illinois  at  Urbana  -  Champaign. 

Griffin,  C.,  (1982).  Manual  of  Built-Up  Roof  Systems.  2nd  Ed.,  McGraw-Hill  Book 
Company,  New  York,  NY. 


60 


USACERL  CP-97/71 


Tri-Service  CADD  Center's  Eiectronic  Bid  Document  Efforts 

by  Ronson  C.  Kiing^* 

Description 

The  Tri-Service  Center  has  developed  a  prototype  showing  how  contract  bids 
could  be  delivered  to  contractors  electronically  on  CD-ROM.  This  protot3^e  is 
the  culmination  of  effort  by  Air  Force,  Army,  and  Navy  personnel  dedicated  to 
the  pursuit  of  electronic  distribution. 

Objective 

The  intent  of  this  project  is  to  replace  existing  paper  reproductions  of  contract 
bid  sets  in  favor  of  electronic  bid  sets. 

Discussion 

Based  on  a  survey  taken  from  NCCOSC  RDTE  DIV  San  Diego’s  contractor 
bidders  list,  the  following  assumptions  can  be  made:  contractors  are  novice 
users  of  personal  computers;  contractors  require  applications  that  are  simple  to 
operate  and  easy  to  use;  a  majority  of  contractors  already  own  and  use  personal 
computers;  the  predominate  operating  system  is  Microsoft®  Windows;  a 
majority  of  contractors  own  laser  printers;  few  contractors  own  plotters;  a 
majority  of  contractors  would  be  interested  in  electronic  contract  bids  if  offered 
through  CD-ROM  or  modem  technology. 

In  evaluating  how  the  Department  of  Defense  can  distribute  contract  bid  sets 
electronically,  the  following  statements  were  made: 

Transition  to  electronic  contract  bid  sets  needs  to  be  gradual.  It  is  the  intention 
of  this  project  to  provide  contractors  with  an  option  of  receiving  bid  sets  by  CD- 
ROM  or  through  the  Internet.  As  contractors  become  more  efficient  with  the  use 
of  computers  and  modem  technology,  the  distribution  of  CD-ROM  may  be 
phased  out.  Contractors  would  be  more  inclined  to  use  electronic  contract  bid 
sets  if  software  were  made  available  to  view  and  print  the  document  royalty- 
free.  Engineering  drawings  will  be  distributed  in  CALS  format,  which  is  consis¬ 
tent  with  DOD  raster  drawing  standards  and  may  also  include  CADD  drawings 
in  their  native  file  formats  (DON,  DWG). 

Due  to  the  complexity  and  detail  of  CADD  drawings,  native  CADD  file  drawings 
tend  to  require  high-end  computers  for  displa3ring  images.  In  general,  con¬ 
tractors  are  not  expected  to  own  high  performance  computers  or  costly  CADD 
packages  for  viewing  engineering  drawings.  However,  CADD  files  may  be 
included  as  part  of  the  contract  bid  set. 


'*  Commander,  U.S.  Army  Engineers  Waterways  Experiment  Station,  ATTN:  Mr.  Ronson  Kung  (CEWES-iM-DA), 
P.O.  Box  631,  Vicksburg,  MS  39181-0631. 
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Raster  drawings  in  CALS  format  provide  compact  files  that  reduce  disk  space 
use  and  speed  transmission  through  the  Internet.  Raster  drawings  also  provide 
contractors  with  an  exact  duplicate  of  paper  drawings  in  electronic  format. 

Reproduction  of  engineering  drawings  by  contractors  should  require  little  techni¬ 
cal  knowledge  when  printing  or  plotting.  Printing  or  plotting  raster  drawings 
will  eliminate  the  need  to  provide  specialized  font  styles,  level/layer  settings, 
scaling,  and  printer  settings. 

Reproduction  services  can  be  provided  by  commercial  printing  services. 

Amendments  will  be  initially  sent  by  paper  or  floppy  diskette.  Amendments  will 
also  be  available  over  the  Internet. 

Use  of  commercial  software  applications  is  highly  desirable  since  technology 
tends  to  outpace  development. 

Contract  specifications  and  clauses  should  be  capable  of  being  viewed,  searched, 
and  reproduced  without  requiring  contractors  to  purchase  a  viewing  program. 
Specifications  and  contract  clauses  will  be  provided  in  PDF  file  format  to 
alleviate  nonstandardization  of  contract  bid  documents.  Currently  DOD  uses  a 
variety  of  software  applications.  Translation  problems  and  version  incompati¬ 
bilities  occur  if  documents  are  distributed  in  native  file  formats.  PDF  is  a 
neutral  file  format  currently  used  by  many  within  the  computer  industry  as  a 
standard  for  distributing  electronic  documents.  Contract  bid  sets  are  a  hybrid  of 
complex  text  and  graphics.  Since  standards  do  not  exist  within  the  text  and 
graphics  commumty,  the  most  popular  standard  file  formats  were  selected — 
PDF  and  CALS  respectively.  Evaluation  of  royalty-free  viewers  capable  of  view¬ 
ing  PDF  and  CALS  file  formats  resulted  in  the  use  of  two  applications,  Adobe’s 
Acrobat  Reader  and  Dataware’s  SourceView.  Acrobat  Reader  was  selected  to 
provide  users  with  capabilities  such  as  zooming,  bookmarks,  links,  and  text 
searches  for  PDF  files;  SourceView  Reader  was  selected  due  to  its  ability  to  view 
CALS  images  and  provide  users  with  zooming,  measuring,  and  linking  features. 

Other  Initiatives 

Currently,  emphasis  of  FACNET  and  EC/EDI  has  been  targeted  in  the  area  of 
small  purchase  acquisition  where  the  majority  of  contracting  actions  reside. 
Traditionally,  small  purchase  contracts  over  $25,000  require  Commerce  Busi¬ 
ness  Daily  (CBD)  announcements.  Use  of  FACNET  will  raise  the  CBD  threshold 
to  $100,000. 

Implementation  of  FACNET  for  many  DOD  agencies  begins  with  processing 
contracting  actions  (request  for  quotations  or  RFQ)  through  the  Standard  Army 
Automated  Contracting  System  (SAACONS).  The  RFQ  is  sent  to  gateways 
where  it  is  translated  into  the  mandated  EDI  message  format  (XI2)  for  the 
Federal  (^vemment.  Through  these  gateways,  an  RFQ  is  sent  to  Network 
Entry  Points  (NEP),  which  are  referred  to  as  Value  Added  Networks  (VANs). 
These  VANs  are  private  industry-owned  services  which  compete  for  vendor/ 
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contractor  business.  VANs  charge  vendor/contractors  for  the  distribution  of 
government  solicitations  and  the  sending  of  bids  back  to  government  agencies  in 
the  appropriate  format. 

Due  to  FACNET  and  EC/EDI’s  emphasis  on  small  pvu*chase  acquisition,  the 
design  of  FACNET  does  not  lend  itself  to  the  electronic  distribution  of  large 
engineering  drawings  and  specifications.  In  addition,  EDI  X12  transaction  set 
number  841  for  technical  information  is  a  standard  that  has  not  been  imple¬ 
mented  and  may  not  be  capable  of  handling  engineering  drawings.  Additional 
testing  will  be  required  before  X12.841  becomes  a  standard  transaction  set  for 
engineering  drawings. 

Concept 

This  initiative  will  provide  contractors  an  option  of  receiving  contract  bid  sets  in 
either  a  CD-ROM  format  or  by  downloading  files  through  the  Internet.  The  CD- 
ROM  version  will  contain  all  data  related  to  the  project  and  include  royalty-free 
viewers  for  quick  access  and  viewing.  Although  no  software  installation  will  be 
required,  contractors  may  want  to  install  the  viewers  onto  their  hard  drives  to 
increase  access  time. 

The  concept  of  downloading  files  through  the  Internet  will  require  access  to  the 
World  Wide  Web.  Using  a  web  browser,  contractors  will  be  given  the  oppor¬ 
tunity  to  query  a  database  of  all  advertised  contracts  submitted  to  a  centralized 
server.  Queries  to  the  central  server  will  result  in  an  HTML  list  of  contract 
advertisements  meeting  the  search  criteria.  Each  advertisement  will  be  linked 
to  the  activities  web  site,  where  contractors  will  be  allowed  to  view  contract 
descriptions,  specifications,  contract  clauses  and  drawings  on-line,  download  all 
associated  files,  or  order  the  CD-ROM. 

Contractors  with  e-mail,  fax,  or  plan  room  access  will  be  allowed  to  subscribe  to 
an  automated  mailing  list  that  will  send  notices  of  contract  advertisements  to 
contractors  based  on  default  search  criteria.  Requests  beyond  the  default  cri¬ 
teria  will  require  contractors  to  search  the  central  server  as  stated  above. 

Benefits 

The  benefits  derived  through  this  project  include  reduction  in  reproduction  costs, 
mailing  costs  and  storage  space  used  to  archive  contract  bid  documents. 

Vision 

The  transition  to  electronic  contract  bid  reqTiires  options  which  allow  contractors 
to  slowly  and  progressively  migrate  towards  electronic  contract  bid.  The  imple¬ 
mentation  of  FACN-ET,  EC/EDI,  and  other  electronic  contract  initiatives  such 
as  this  project  will  continue  to  progress  and  be  refined  as  changes  in  technology 
occur.  Eventually,  lessons  learned  through  these  initiatives  will  serve  as  a  com¬ 
mon  thread  for  development  of  a  comprehensive  electronic  contract  bid  process 
capable  of  providing  contractors  with  a  centralized  contract  network. 


USACERL  CP-97/71 


63 


Requirements 

The  CD-ROM  delivery  of  electronic  contract  bids  will  require  either  Windows 
3.1™,  Windows  NT™,  or  Windows  95™.  The  recommended  hardware  require¬ 
ments  are  486/33Mhz  with  16MB  RAM  or  better  and  a  Super  VGA  monitor  (800 
X  600  resolution).  The  absolute  minimum  hardware  requirements  are  386  with 
8MB  RAM  and  a  VGA  monitor  (640  x  480  resolution).  Please  note,  speed  and 
performance  depend  on  processing  power  and  the  amount  of  RAM  available. 

Internet  access  may  be  required  to  receive  on-line  electronic  contract  bids.  File 
transfer  speeds  depend  on  the  speed  of  modems  used  by  the  receiver  and  sender. 
On-line  access  may  require  subscriptions  to  plan  rooms,  VANs,  or  service 
providers. 

Test  Sites 

Contact  one  of  the  persons  listed  below  for  the  latest  information  on  test  sites: 

J.  Justin  Taylor,  Corps  of  Engineers,  HQUSACE-MP-ES,  20  Massachusetts  Ave. 
NW,  Washington  DC  20314-1000.  Phone:  601-634-2152,  Fax:  601-634-3448,  e- 
mail:  taylor@exl.wes.army.mil. 

David  Skar,  Naval  Facilities  Engineering  Command  HQ,  200  Stovall  Street, 
Alexandria,  VA  22332.  Phone:  703-325-7360,  Fax:  703-325-2261,  e-mail: 
djskar@ha.navfac.navv.mil. 

Captain  Mike  Stimson,  HQ  AFMC/CECC,  4225  Logistics  Ave.,  Suite  #7,  Wright 
Patterson  AFB,  OH  45433-5746.  Phone:  (513)  257-7214,  Fax:  (513)  257-1830,  e- 
mail:  stimson@afhicce.wpafb.mil. 
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The  Modular  Design  System  (MDS) 

by  Eric  Griffith'’^ 

Outline 

What  is  MDS? 

Who  is  using  it? 

How  is  it  going  to  be  managed? 

What  are  future  developments? 

Modular  Design  System 

Developed  by  Louisville  District  as  the  Center  of  Standardization  for  Army 
Reserves.  MDS  is  a  component  module  design  approach  for  defined  facility 
types.  Modules  are  configured  within  defined  grids.  It  is  developed  within 
MicroStation.  First  Facility  Type  -  Army  Reserve/National  Guard  (AR/NG) 
Training  Centers.  System  Delivery  -  15  January  1996.  It  has  been  imder 
development  for  3+  years. 

Three  Module  Types 

Fully  Designed  Modules:  The  modules  are  completely  and  independently  de¬ 
signed  for  a  specific  programmatic  function. 

Planning  Modules:  The  modules  track  programmatic  functional  criteria,  but  the 
modules  have  not  been  completely  designed. 

Structural  Modules:  Structural  modules  define  structural  spacing  and  control 
member  selection.  A  limited  number  of  sizes  are  available — 32  x  32  pre¬ 
dominately. 

Information  Row 

A  “Layout”  of  modules  is  created  on  a  selected  grid.  The  layout  is  converted  to 
Architectural  Drawings.  Doors,  windows,  and  wall  finishes  are  user  input. 

Now,  the  other  disciplines  can  proceed.  Most  of  the  disciplines  do  not  design 
systems,  but  aid  in  docmnentation  (i.e.,  duct  sizes  are  not  determined,  but  tools 
are  available  to  connect  diffusers,  supplies,  and  returns).  Drawings  are  com¬ 
pleted  in  native  MicroStation. 


"  Commander,  U.S.  Army  Construction  Engineering  Research  Laboratories,  P.O.  Box  9005,  ATTN:  Mr.  Eric 
Griffith  (CECER-PL-E),  Champaign,  IL  61826-9005. 
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MDS  Products 

DD  1391  -  Documents  for  Congressional  Approval,  Construction  Documents 
(Plans  and  Specifications)  75%,  Cost  estimates  at  each  stage,  Fumitme  pur¬ 
chasing  requirements. 

Who  Is  Using  It  Today? 

•  Louisville  District 

•  Two  Indefinite  Delivery  Contractors 

•  Various  State  National  Guard  Agencies 

•  MDS  Supports  75%  of  the  Design  Production  activities  for  AE/NG 

MDS  Future  Vision 

•  The  facility  delivery  system  for  standard  facilities  within  the  Army 

•  The  facility  delivery  system  for  standard  facilities  within  the  DoD 

•  The  facility  delivery  system  for  “recurring  buildings”  within  the  Federal 
Government 

•  Re-engineer  the  federal  facility  delivery  process 

MDS  Defined  as  APIs 

MDS-API  Core:  The  core  program  functionality  including  data  representations, 
system  software,  and  CAD  program  interfaces. 

Engineering  Discipline-API  (ED-API):  Builds  from  the  MDS-API  to  meet  the 
functional  requirements  of  each  technical  discipline  as  needed  by  FT-APS.  This 
includes  data  definition  and  routines  specific  to  the  ED-API. 

Facility  Type-Application  Program  (FT-AP):  Utilizing  the  ED-APIs,  an  FT-AP  is 
developed  to  meet  the  needs  of  specific  facilities.  This  includes  data  definition 
and  FT-AP  specific  routines. 

Organizational  Roles  for  MDS  Activities 

•  MDS  Proponent  -  HQUSACE  -  CEMP-EA 

•  Program  Agent  -  Tri-Services  CADD  Center 

•  Technical  Agent  -  USACERL 

•  Engineering  Discipline  Committees 

•  Facility  Type  Application  Committees 

MOS  Within  the  Corps 

•  Module  Creation  Program 

•  New  Facility  Types:  Vehicle  Maintenance  Shops,  Barracks 

•  Build  support  outside  the  Army:  Bureau  of  Indian  Affairs 

•  Contractor  has  asked  to  utilize  MDS  on  an  unrelated  government  project 
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MDS  Cooperative  Research  and  Development  Agreement 

•  Partners:  IdeaGraphix/Softdesk,  Bentley  Systems,  JMGR,  and  Building  Sys¬ 
tems  Design 

•  Open  Collaborative  Engineering  Framework:  Semantic  Object  Representa¬ 
tions,  Plug  and  Play  Architecture,  Conflict  Processes,  Repositories 

•  Design  Review  an  important  issue  for  future  work 
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7  Summary 

Toward  a  Design  Review  Lessons-Learned  System 

by  Donald  K.  Hicks,  Jeffrey  G.  Kirby,  and  E.  William  East“ 

In  their  role  as  Constimction  Manager  for  the  U.S.  Army,  U.S.  Air  Force,  and 
other  Federal  Agencies,  the  U.S.  Army  Corps  of  Engineers  is  constantly  striving 
to  improve  the  quality  of  the  Facility  Delivery  Process  and  the  product— the 
facility  that  is  delivered  to  their  customers.  Essential  to  this  effort  is  the  design 
review  process  and  the  effective  capture  and  use  of  the  lessons  learned.  On  23- 
25  January  1996,  the  U.S.  Army  Construction  Engineering  Research  Labora¬ 
tories  (USACERL)  hosted  the  “Design  Reviewer's  Support  Environment  Project 
Steering  Committee  Meeting”  in  Champaign,  IL. 

The  purposes  for  this  meeting  were;  (1)  to  determine  the  current  state  of  the 
practice  of  design  review  tools,  primarily  but  not  exclusively  within  the  Corps  of 
Engineers,  (2)  to  identify  promising  directions  for  future  research  efforts  into 
design  products  and  systems,  and  (3)  to  identify  members  of  the  steering  com¬ 
mittee  who  would  be  willing  to  evaluate  and  jointly  develop  future  design  review 
tools.  This  paper  summarizes  the  events  of  this  meeting. 

Attending  the  meeting  were  Corps  of  Engineers  personnel  from  all  organiza¬ 
tional  levels  who  were  responsible  for  ensuring  that  the  facility  delivery  process 
and,  specifically,  the  construction  plans  and  specifications  are  of  the  highest 
possible  quality.  Private  sector  construction  document  code  review  professionals 
and  graduate  student  research  assistants  also  attended.  A  list  of  persons 
attending  this  meeting  is  provided  on  p  68. 

The  meeting  was  organized  in  three  distinct  segments.  First,  attendees  spoke 
describing  design-review  and  design-review-oriented  lessons-leamed  systems 
and  processes  currently  used  across  the  Corps  of  Engineers.  Prototype  systems 
to  support  the  design  review  and  lessons-leamed  systems  were  presented.  Inno¬ 
vative  systems  . developed  outside  of  the  Corps  of  Engineers  were  also  presented 
to  illustrate  specific  capabilities  and  interests.  Then  the  group  developed  a 
listing  of  prioritized  major  issues,  believed  to  be  the  most  significant,  facing  the 
design  profession  and,  more  specifically,  the  Corps  of  Engineers’  interest  in 
design  review.  And  finally,  the  group  identified  what  actions  were  required  to 
facilitate  the  implementation  of  solutions  and  who  would  be  the  most  appro¬ 
priate  initiator  and  proponent  of  each  action  item. 


"  All  from:  Commander,  U.S.  Army  Construction  Engineering  Research  Laboratories,  P.O.  Box  9005,  ATTN: 
CECER-PL-E,  Champaign,  IL  61826-9005. 
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Name 

Affiliation 

e-address 

Telephone 

Drew  Anderson 

Omaha  District 

drew.I.anderson 
(2)MRO01  .usace.army.mil 

402-221-4454 

Johnny  Baggette 

South  Atlantic  Division 

Margie  Crumley 

Omaha  District 

402-221-3979 

Bill  East 

CERL 

b-east(g)cecer.army.mil  . 

217-373-6710 

John  Hart 

Ohio  River  Division 

john_hart@smtp.ord.usace 

.army.mil 

513-684-3803 

Terry  Houghton 

HQ 

Terence.Houghton@inet. 

HQ.usace.army.mil 

202-761-0427 

Blaine  Kemsiey 

Albuquerque  District 

Blaine. R.Kemsley@swa01 . 
usace.army.mil 

505-254-3343 

Ronson  Kung  . 

WES 

kungr@ex1  .wes.army.mil 

601-634-3181 

Carl  Mileff 

CMA  &  Associates 

cmileff@aol.com 

209-226-0205 

Glenn 

Rasmussen 

CERL 

g-rasmussen 

@cecer.army.mil 

217-373-7537 

Norman  Sams 

Alaska  District 

Norman.D.Sams@NAP01 . 
usace.army.mil 

907-353-7556 

Stephen  Stoner 

ARMS 

Stoner@usace.army.mil 

916-557-7676 

Justin  Taylor 

HQ 

jtaylorj@ex1  .wes.army.mil 

202-761-1246 

By  way  of  an  introduction  to  the  meeting,  six  possible  domains  of  interest  were 
presented  as  a  context  into  which  the  meeting  outcomes  could  be  framed.  They 
were:  (1)  Operating  Systems  for  work  groups,  graphical  users  interface  (GUI), 
and  mrdtimedia,  (2)  Electronic  Bid  Documents  to  mark-up  reviews  and  for 
printing  cost  avoidance,  (3)  Lessons-Leamed  for  daily  uses  and  after  action 
documentation,  (4)  Automated  Review  Management  System  (ARMS),  which 
provides  work  group  support,  suspense  tracking,  and  routing,  (5)  Internet 
(WWW)  for  information  access,  work  group  support,  and  data  depository,  and  (6) 
Computer-Aided  Design  and  Drafting  (CADD)  systems  for  collaborative  environ¬ 
ment  and  lines  of  communication.  The  challenge  to  the  meeting  participants 
was  to  integrate  any  of  the  possible  solutions  with  the  above  cited  resources 
while  realistically  measming  the  solutions  against  the  resource  limitations, 
changing  workloads,  and  the  requirement  for  faster  design  review  cycle  time. 

Summary  of  Presentations 

The  following  topic  sessions  were  conducted  during  the  meeting.  In  general,  all 
of  the  attendees  presented  the  results  of  their  efforts  or  the  results  of  their 
organization  at  one  or  more  of  the  sessions.  The  session  topics  were  developed 
with  the  intent  to  further  define  the  purposes  of  the  meeting. 

Automated  Review  Management  System  (ARMS):  ARMS  is  a  Corps  of  Engineers 
system  developed  to  support  the  management  of  the  design  review  process  as 
practiced  by  the  Corps  of  Engineers.  The  Sacramento  District  office  is  the  center 
for  technical  expertise  for  ARMS.  ARMS  users  discussed  current  use  of  the 
system.  Several  District  offices  and  USACERL  have  developed  programs  to 
enable  the  generation  of  “off-line”  review  comments  that  then  can  be  forwarded 
to  the  ARMS  central  computer.  There  were  presentations  of  the  systems  by  their 
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users/developers.  These  programs  were  discussed  by  the  committee.  ARMS 
implementation  barriers  were  discussed. 

Reviewer  s  Assistant  and  the  Lessons-Learned  Generator'.  The  Corps  of  Engi¬ 
neers  recognizes  the  need  for  design  review  support  and  directed  USACERL  to 
execute  Research  and  Development  to  meet  this  need.  The  results  of  USA¬ 
CERL  s  effort  has  been  the  development  of  a  design  review  support  system,  the 
Reviewer’s  Assistant,  and  a  lessons-leamed  system,  the  Lessons-Learned 
Generator.  Together  they  collect,  abstract,  and  compile  commonly  referenced 
review  comments.  The  two  systems  have  been  developed  to  support  the  design 
review  process  commonly  used  by  Architect/Engineer  (A/E)  offices.  The 
Reviewer  s  Assistant  is  an  integrated  text  editor,  database,  and  commimication 
software  package  in  which  reviewers  create,  store,  and  query  for  applicable 
review  comments.  The  Lessons-Learned  Generator  performs  a  statistical  analy¬ 
sis  of  the  usage  patterns  of  review  knowledge  stored  in  the  Reviewer's  Assistant 
databases,  performs  an  abstraction  process  on  commonly  referenced  review  com¬ 
ments,  and  compiles  them  into  a  lessons-leamed  database.  Together  the  two 
systems  form  a  powerful  and  flexible  tool  for  design  reviewers. 

Lessons-Learned  Initiatives:  The  Corps  of  Engineers  recognizes  that  there  must 
be  a  continuing  evaluation  of  the  functional  responsiveness  and  technical  per¬ 
formance  of  the  Corps  practices  of  design,  constructioii,  and  post-construction 
for  constructibility,  engineering  and  technical  sufficiency,  life-cycle  cost  perfor¬ 
mance,  lessons-leamed  techmeal  feedback,  and  compliance  with  current  design 
and  constmetion  criteria.  The  Corps  has  established  various  requirements  and 
methods  .to  obtain,  evaluate,  and  incorporate  recommended  changes  in  design 
and  constmetion  policy,  guidance,  and  criteria.  In  addition  to  the  policy  estab¬ 
lished  by  the  Corps,  several  District  offices  have  initiated  local  level  programs  to 
capture  and  use  lessons  learned.  The  question  of  whether  the  lessons-leamed 
system  should  be  a  separate  system  or  integrated  with  other  systems,  such  as 
the  Reviewer  s  Assistant,  was  discussed.  All  of  the  participants  strongly 
believed  that  capture  and  use  of  lessons  learned  was  a  key  component  of  the 
Corps’  ability  to  deliver  quality  facilities  that  meet  the  user’s  needs. 

Other  Corps  of  Engineers  Review  Process  Reports:  Several  participants  reported 
on  how  their  organization  conducts  and  manages  design  reviews.  Since  each 
Corps  District  and  Division  office  operates  autonomously,  this  was  of  particular 
interest  to  the  group.  Some  offices  have  not  as  yet  opted  to  implement  any  of  the 
available  automation  support,  and  this  discussion  was  of  value  to  their  decision¬ 
making  ptocess. 

Electronic  Bid  Documents:  With  the  advent  of  electronic  bidding  documents  and 
the  fact,  that  several  district  offices  are  testing  them  on  specific  projects,  the 
group  discussed  the  potential  of  using  this  media  to  its  maximum  in  the  design 
review  process.  The  process  of  issuing  electronic  bid  documents,  controlling  the 
versions,  security  issues,  and  document  management  were  discussed.  All 
recogmzed  that  the  new  media  posed  new  challenges  that  needed  to  be 
addressed  in  order  to  successfully  manage  the  task  and  comply  with  Federal 
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Acqmsition  Regulations.  It  was  also  concluded  by  the  group  that  the  benefits  of 
the  electronic  document  media  outweighed  the  negative  aspects. 

Design  Review  Related  Prototype  Systems:  Several  new  commercial  and  Corps 
developed  systems  were  discussed  by  the  group  with  the  specific  interest  issue  of 
how  they  could  support  the  design  review  process  as  well  as  customer  satis¬ 
faction.  “Lotus  Notes”  was  demonstrated  to  the  group  and  discussions  followed 
as  to  the  practical  application  of  the  system  to  the  design  review  process.  The 
Corps  of  Engineers  developed  “Modular  Design  System”  (MDS)  was  demon¬ 
strated  and  discussed  also.  The  feasibility  and  functional  ability  of  computer- 
assisted  design  reviews  were  discussed.  How  they  might  support  the  process, 
their  functionality,  and  the  current  state  of  development  were  discussed.  The 
group  concluded  that  the  future  holds  much  promise  in  this  area. 

Design  Review  Systems,  “The  Next  Generation”:  Given  that  the  group  recognized 
the  need  to  further  enhance  the  design  review  process,  thoughts  were  directed  to 
the  possibility  of  capitalizing  on  recent  advances  in  technology  as  well  as  the 
Corps’  emphasis  on  improving  the  Facility  Delivery  Process  from  the  customers 
unique  point  of  view.  Specific  next  generation  tools  identified  for  discussion 
included:  new  hardware/software  platforms,  groupware/routing  systems,  screen 
layout  and  configuration  with  graphical  user  interfaces,  electronic  document 
mark-up  capabilities,  design  data  as  well  as  graphical  (drawings)  data,  the  most 
effective  and  efficient  means  of  transferring  to  the  next  generation,  adaptable 
commercial  systems  in  use,  regulatory  requirements,  the  World  Wide  Web 
Internet  systems  as  a  supporting  capability  of  design  review,  and  efficient  and 
effective  technology  transfer  through  adaptive  and  intuitive  systems. 

Summary  of  the  Breakout  Groups 

As  a  group  of  the  whole,  the  committee  discussed  what  were  the  major  issues 
that  needed  to  be  addressed  for  the  “Next  Generation  System”  for  design  review 
support.  An  extensive  list  of  topics  was  developed  and  discussed  in  an  unre¬ 
stricted  free  flow  environment.  The  committee  then  divided  into  two  smaller 
subgroups  (Group  A  and  Group  B)  and  selected  topics  of  interest  from  the  full 
list  of  issues.  The  two  groups  were  then  tasked  to  prioritize  their  topics  in  terms 
of  importance  and  to  develop  potential  solutions,  recommendations,  and  sug¬ 
gestions  for  the  resolution  of  their  highest  ranking  topics  as  time  permitted. 

Each  group  then  fiirther  defined  and  refined  their  topics  to  the  point  of  coh- 
sensus,  consolidated  where  possible,  and  prioritized  the  topics,  most  important 
to  least.  The  groups  then  discussed  and  analyzed  the  topics,  developed  solutions 
and  recommendations,  and  made  preparations  for  presentations  to  the 
committee  as  a  whole. 

Group  A  reported  their  top  five  issues  as:  (1)  a  shared  lessons-leamed  system 
with  user  fifiendly  interface,  (2)  improve  the  design  review  process,  (3)  provide 
leading  edge  solutions,  not  bleeding  edge,  (4)  provide  within  the  design  review 
process  a  mechanism  for  Architect/Engineer  evaluations  based  on  the  quality  of 
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the  documents,  and  (5)  provide  a  feedback  system  that  addresses  the  change 
order  process. 

Group  B's  top  four  issues  were:  (1)  user  friendliness  (software  ergonomics),  (2) 
lesson-leamed  feedback  mechanism,  (3)  simplification  of  the  design  review 
process,  and  (4)  interface  with  industry.  Group  B  reported  on  their  top  issues  in 
terms  of  (a)  technical  issues,  (b)' adoption  by  the  user  community,  (c)  approval, 
(d)  process  issues,  and  (e)  industry  partnership  issues. 

Action  Items 

As  a  result  of  the  committee’s  review  of  the  existing  design  review  process 
condition  and  the  brainstorming  session  on  what  the  next  generation  of  design 
review  systems  cotJd  be,  the  following  is  a  list  of  the  actions  needed  in  order  to 
advance  the  effective  design  review  process  and  product. 

1.  Recommend  that  ARMS  is  actively  used  by  all  elements  of  the  Corps  of 
Engineers,  including  the  Civil  Works  Directorate.  The  tools  are  there,  use 
them — ^ARMS-^,  Reviewer's  Assistant,  and  PC  ARMS. 

2.  Create  a  prototype  “Lessons-Leamed”  World  Wide  Web  (WWW)  site  with  e- 
mail  to  promote  the  exchange  of  information  among  all  design  reviewers. 
Private  sector  use  of  this  data  may  also  be  explored. 

3.  Create  a  WWW  site  for  the  Steering  Committee  for  the  exchange  of 
information  among  committee  members. 

4.  Develop  system  design  guidance  that  woidd  ensme  that  future  review 
systems  would  be  modular  and  therefore  compatible  with  each  other  and 
with  future  systems. 

5.  Proceed  with  the  cooperative  research  and  development  agreement  to 
develop  a  plan  checking  system. 

6.  Develop  a  system  or  mechanism  to  provide  design  references  and  criteria 
electronically  to  the  design  reviewers. 

7.  Proceed  with  the  development  of  the  capability  to  electronically  “mark-up” 
design  drawings. 

8.  Improve  connectivity  of  the  remote  offices  by  design  review  community  to  the 
WWW. 

Conclusions 

The  consensus  of  opinions  of  the  conference  attendees  was:  (1)  that  the  need  for 
efficient  and  effective  design  reviews  of  construction  documents  are  more  impor¬ 
tant  now  than  ever,  and  they  are  essential  for  customer  satisfaction,  (2)  the 
Internet  offers  opportunities  in  which  all  of  the  facility  acqviisition  process 
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participants  can  become  connected,  and  with  this  connectivity  the  commimica- 
tion  process  will  be  greatly  facilitated,  and  (3)  to  remain  a  viable  participant  in 
the  process  there  must  be  a  strong  commitment  to  using  advanced  technologies 
to  improve  the  way  in  which  we  provide  our  services  to  our  customers. 

The  group  recognizes  that  there  are  at  least  two  remaining  issues  to  be  resolved: 
(1)  whether  the  design  review  process  will  remain  as  individual  reviewers 
generating  their  comments  without  collaboration  or  interaction  or  will  it  become 
a  “virtual  design  conference”  and  (2)  will  the  ultimate  tool  become  embedded  in 
the  design  review  process  or  will  it  be  a  reference  material  that  is  used  at  the 
discretion  of  the  reviewer? 
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